PHP Security: disable error_reporting() NOW

The arrival of amazing malware

In one of the servers which I’ve been securing against malware, an amazing backdoor PHP script was found:

<?php error_reporting(0); ${"x47Lx4fBx41x4cx53"}["x72x6dx63vfy"]="bx6fx74x5fx75x73ers";${"x47x4cx4fBx41x4cx53"}["x67x74lx76x68x75x6e"]="x62x6ft_ix70s";${"x47x4cx4fBx41x4cx53"}... lots of obfuscated code follows..

Why was it undetected by malware scanners? And how can we detect this kind of malware ourselves?

I won’t go into much details what the actual code does. In short, it allows hackers to run any commands on the compromised servers.
The code is highly obfuscated and there were dozens of similar scripts implanted in multiple directories of the website in question.

Take note at the opening code: error_reporting(0). It tries to turn off any errors from being written to PHP error log or displayed to browsers. And it succeeds.

php_admin_flag and error_reporting(0)

Supposedly, you did the right thing in configuring your PHP-FPM pool, e.g.:

php_admin_value[error_reporting] = E_ALL & ~E_NOTICE & ~E_STRICT & ~E_DEPRECATED

You’d expect that error_reporting(0) function calls will have no effect on the configured secure value of logging level. You’re wrong. At least as of PHP 7.0, there’s a long-standing bug which makes php_admin_value[error_reporting] useless.

With 7.0 <= PHP <= 7.?, any script can call error_reporting(0) and bypass whatever secure log level you have chosen.

You can see whether your specific PHP version is affected by running a simple test script. Provided you have placed the sample php_admin_value[error_reporting] directive in your PHP-FPM pool, create a script:

<?php
var_dump(error_reporting());
error_reporting(0);
var_dump(error_reporting());

Affected PHP versions will emit int(22519) int(0), meaning that error_reporting(0) is allowed to override master PHP configuration value.

Surely enough, if hackers want to use your server as a tool, they want to keep their profile low. This is especially true for cryptojackers who want to leverage the power of your server to mine Monero. They want to do it as long as possible and stay undetected for as long as possible.

And the error_reporting PHP function is just there for their benefit. No matter what secure configuration PHP 7 has on the server level, the error_reporting(0) can override it and completely silence errors in affected scripts.

I can’t think of a good use for error_reporting function at all!. Apart from the mentioned malicious intent, error_reporting(0) is also a darling of bad coders trying to silence their code from emitting errors and warnings. This just makes things hard to troubleshoot on live servers.

If you know how to configure servers properly and have already specified the necessary logging level via error_reporting php.ini configuration directive, then you don’t need error_reporting(...) function at all.

Disable error_reporting(…)

Unfortunately, with the aforementioned PHP 7 bug, any script can override the configured error level, by just calling error_reporting(0). The only way to stop this is to disable the function altogether.

Make sure that your php.ini is configured with:

disable_functions=error_reporting

This will emit a security warning for scripts that use the function (they won’t fail). There, one change got us 2 things:

• The malware can’t hide so easily as we have raised the chances of it exposing itself via PHP error log
• We know who’s only trying to appear as a good coder by having PHP run with mouth shut about their unfixed bugs

But WordPress…

WordPress is trying to outsmart us and their developers think that we don’t know how to configure our servers. Even with default configuration (with WP_DEBUG off), you’d notice:

error_reporting() has been disabled for security reasons in wp-load.php on line 24
error_reporting() has been disabled for security reasons in load.php on line 333

You can’t do much about this: ignore the warnings or comment out those lines with ‘//’ to keep your logs clean (and also make things a little faster by running 2 fewer lines of PHP. Micro optimization maniac detectado 🙂

Oh wait, you can actually use an automated patch plugin. Installable via CLI:

wp plugin install https://github.com/GetPageSpeed/wp-error-reporting-patch/archive/master.zip --activate

If you go “ignore the warnings” route (keep WordPress files intact), you may want to filter WordPress cron errors.

@ Not so fast @

PHP has one built-in error handling operator which is @. If you prepend anything to a code with this sign, the error will be silenced.

There are basically 2 solutions to unsilence all pieces of code that make use of it.

First, is PHP based. In your bootstrap PHP file, set a custom PHP error handler ( via set_error_handler() ).

Second, is to install and configure a PHP extension which will stop the scream operator from working.

For PHP < 7.0, this scream PHP extension. To install it, run yum install php-pecl-scream. After the extension is installed, configure it in php.ini:

scream.enabled = On

For PHP >= 7.0, you can use XDebug extension. Once installed, configure it in php.ini with:

xdebug.scream=1

Note that the value of 1 disables scream operator, so the setting value is counter-intuitive. From XDebug documentation:

If this setting is 1, then Xdebug will disable the @ (shut-up) operator so that notices, warnings, and errors are no longer hidden.

On another note about XDebug – you can use to prevent error reporting calls from hiding errors as well. Place this in your php.ini:

xdebug.force_display_errors = 1;
xdebug.force_error_reporting = -1;

Return of the malware

Fast forward to May 14, 2018. And I’m dealing with another malware that uses error_reporting to hide itself. This time it has made some evolution: there are markers with an ID of the “hack”. The files have the same signature at the beginning. Some of the files showed signs of double penetration 😀 :

    <?php /*564794552*/ error_reporting(0); @ini_set('error_log',NULL); @ini_set('log_errors',0); @ini_set('display_errors','Off'); @eval( base64_decode('blah 
    blah lots of encoded stuff')); @ini_restore('error_log'); @ini_restore('display_errors'); /*564794552*/ ?><?php /*8793453*/ error_reporting(0); ... /*8793453*/ ?>

Amazing. This could have to effect been hacked by 2 different people using the same malware. On that particular server, they had an old PHP and Apache. Still, there was no error anywhere as they haven’t setup secure php_admin_value for error log level. Their site failed with a 500 error.

However, in some files, the malware manifested itself with:

Namespace declaration statement has to be the very first statement in the script in …

The number of affected PHP files count was topping 10K. That is, the malware put itself into every single PHP file out there. How do you find and weed out all these malware strings from your files?

Find infected file with:

grep -R --include=*.php "error_reporting(0)" 

And further you can count them with:

grep -R --include=*.php "error_reporting(0)"  | wc -l

To clean then up, use the sed command line program. The following will replace all the marked PHP code blocks like <?php /*8793453*/ ... /*8793453*/ ?> while keeping a copy of “hacked” files with .virus extension:

find . -iname "*.php" -exec sed -i.virus --regexp-extended 's@<?php /*[0-9]+*/.*/*[0-9]+*/ ?>@@g' {} ;

Needless to say, you have to take some measures on having the hack not happen again:

• Upgrade server software
• More importantly, secure your open source apps, like WordPress (update core and plugins)

Conclusions for admin:

1. Disable the silencing operator:

Install the PHP extensions to control it:

sudo yum install php-pecl-scream 

Edit your php.ini and put:

scream.enabled = Off

2. Disable error_reporting

Edit your php.ini and put:

disable_functions=error_reporting

3. Ensure non-overridable log settings

As you might have noted, the malware in question tried to override some settings via ini_set. You can disable this function as well, but many apps use ini_set for their function.
So you might want to simply enforce the important security-related settings in your web server configuration. Use of php_admin_flag or php_admin_value directives is highly recommended while configuring log settings for your websites.

Conclusion for hackers:

• Use @ silencing operator for every function call In the latest example they were using it, but for some reasons error_reporting call itself was not subjected to @. Other from that things were advanced enough to keep current log level and make sure that non-hacked related code keeps its current logging level. In a way, this allowed the malware to shamelessly infect every PHP file on the server.
• Try not to break the code in a way that you will manifest yourself