Magento 2 cron jobs

Goal: Setup PROPER Magento 2 cron jobs

The following are suggested crons from Magento 2 documentation. Assuming that your Magento 2 installation is in /var/www:

* * * * * /usr/bin/php /var/www/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/var/log/magento.cron.log
* * * * * /usr/bin/php /var/www/update/cron.php >> /var/www/var/log/update.cron.log
* * * * * /usr/bin/php /var/www/bin/magento setup:cron:run >> /var/www/var/log/setup.cron.log

This is wrong for at least a couple of reasons.

Magento 2 crons and memory

Cron tasks usually require more RAM than web scripts. Magento 2 recommended PHP memory_limit is 768M.

So we want to remove memory limitations for Magento cron jobs only. The proper crons now become:

* * * * * /usr/bin/php -d memory_limit=4G /var/www/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/var/log/magento.cron.log
* * * * * /usr/bin/php -d memory_limit=4G /var/www/update/cron.php >> /var/www/var/log/update.cron.log
* * * * * /usr/bin/php -d memory_limit=4G /var/www/bin/magento setup:cron:run >> /var/www/var/log/setup.cron.log

Now your Magento 2 cron jobs will run without any memory limitation by PHP. Great.

Magento 2 crons and fatal PHP errors

It may happen to you as it happened to me. A severely bad coded, beta quality plugin caused failed cron runs with absolutely no logging.
This is because by default PHP error_log is empty and the cron definitions we had so far only redirect standard output to the log files.

Let’s make things right:

* * * * * /usr/bin/php -d memory_limit=4G /var/www/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /var/www/var/log/magento.cron.log
* * * * * /usr/bin/php -d memory_limit=4G /var/www/update/cron.php 2>&1 >> /var/www/var/log/update.cron.log
* * * * * /usr/bin/php -d memory_limit=4G /var/www/bin/magento setup:cron:run 2>&1 >> /var/www/var/log/setup.cron.log

You’ll notice that we’ve added 2>&1 to each cron job. This redirects error output to standard output. So everything included fatal PHP errors are now being logged to corresponding files.

Alert emails?

Cron has a feature that is annoying to some and wonderful to others. That is, sending an email when a cron job outputs something. This is good to be alerted when something doesn’t work as expected. So you may want to modify your cron jobs, to output only errors (those will be sent by email) while still logging everything:

* * * * * /usr/bin/php -d memory_limit=4G /var/www/bin/magento cron:run 2> >(tee -a /var/www/var/log/magento.cron.log >&1) > >(grep -v "Ran jobs by schedule" >> /var/www/var/log/magento.cron.log) 
* * * * * /usr/bin/php -d memory_limit=4G /var/www/update/cron.php 2> >(tee -a /var/www/var/log/update.cron.log >&1) >> /var/www/var/log/update.cron.log
* * * * * /usr/bin/php -d memory_limit=4G /var/www/bin/magento setup:cron:run 2> >(tee -a /var/www/var/log/setup.cron.log >&1) >> /var/www/var/log/setup.cron.log

This will log both errors and regular output to the log files. And it will also output errors to standard output, which cron will email to us. How in the world does this work? 🙂

Let’s review the first defined cron job bits:

• 2> >(tee -a /var/www/var/log/magento.cron.log >&1): this redirects stderr (2>) to a command substitution. The command substitution has a format of >(...). In our case it uses tee to log to file as well as output to stdout.
• Note that the order of redirections is important. Read left to right: we now output stderr to current destination of stdout and file. And that’s it for stderr. It won’t pick up on subsequent redefinition of stdout destination. That got me puzzled when initially understanding how redirections order works. The keyword here is current.
• Next bit > >(grep -v "Ran jobs by schedule" >> /var/www/var/log/magento.cron.log) redirects stdout to a command substitution: which filters for a message that we never want to log, and logs everything else the log file and outputs to stdout.

Now:

• We will get an email once a cron fails with an error
• We can easily expand the above crons to log errors to a different file.

The command substitution parts of our crons is supported by bash and not sh. So if you’re in doubt which is the one your cron uses, specify it in your cron. Just add a line at the top of all the cron definitions:

SHELL=/bin/bash

Speeding up Magento 2 cron with PHP OPCache

With PHP 7.0 and above, the file system can be used for storing compiled PHP scripts. This allows us to increase performance for Magento 2 cron jobs by up to 100%. Simply create an .opcache directory where compiled scripts are going to be saved and adjust crons with additional parameters -d opcache.file_cache_only=1 -d opcache.enable_cli=1 -d opcache.file_cache=/var/www.opcache:

* * * * * /usr/bin/php -d opcache.file_cache_only=1 -d opcache.enable_cli=1 -d opcache.file_cache=/var/www.opcache -d memory_limit=4G /var/www/bin/magento cron:run 2> >(tee -a /var/www/var/log/magento.cron.log >&1) > >(grep -v "Ran jobs by schedule" >> /var/www/var/log/magento.cron.log) 
* * * * * /usr/bin/php -d opcache.file_cache_only=1 -d opcache.enable_cli=1 -d opcache.file_cache=/var/www.opcache -d memory_limit=4G /var/www/update/cron.php 2> >(tee -a /var/www/var/log/update.cron.log >&1) >> /var/www/var/log/update.cron.log
* * * * * /usr/bin/php -d opcache.file_cache_only=1 -d opcache.enable_cli=1 -d opcache.file_cache=/var/www.opcache -d memory_limit=4G /var/www/bin/magento setup:cron:run 2> >(tee -a /var/www/var/log/setup.cron.log >&1) >> /var/www/var/log/setup.cron.log

Now we have proper and faster Magento 2 cron jobs setup. You can checkout Paperttrail for cloud-based aggregated logging facility for your Magento log files.