Magento and ElasticSearch? How about Sphinx!

Magento 2 and its “poor” choice of technologies

Magento 2 as an E-Commerce framework, is capable of empowering very large and high-traffic webshops.
It is designed in a way that critical functions like search can scale to dozens of servers.

But Magento 2 absolutely doesn’t want you to run it on a single medium-size server, by default. How?

ElasticSearch is the default engine in Magento 2 as of recently. It may be a great software, it may scale to many servers, but being Java-based, it has quite some memory requirements,
Those requirements will push you into buying a very powerful server. However, there’s little chance that your webshop has the traffic the size eBay or YouTube.

There’s really no point in using memory-hungry ElasticSearch when you’re running any small-size business.

Magento 2 is developed without understanding the simple critical fact:

The winning majority of Magento 2 installations are not the size of actual eBay and are seeking to run everything in one mid-sized server.

The defaults should allow Magento 2 to run on a single server. But now this isn’t true unless you’re running a server with a lot of RAM.

If you do have at least 32GB RAM on your server, ElasticSearch is actually great and you should use it.
Otherwise, you will soon find that things are way too slower than they should be.

And it’s often not because of the slow server, but because Magento could embrace different technologies to make it more efficient in all-in-one server scenarios.

Yes, Magento 2 could have defaulted to a different, more efficient search engine too. There’s a way to fix the unwelcome Elasticsearch default!

The following instructions are applicable to CentOS/RHEL 6, 7, and 8.

Sphinx search plugin for Magento 2

What’s more efficient than the Java-based Elasticsearch? Virtually anything that is based on C/C++.

The Sphinx search engine not only satisfies that requirement but also has tremendously small memory and CPU footprint when compared to ElasticSearch.

To empower Magento 2 with the ability to use the Sphinx search engine, you must first grab the corresponding search plugin.

While being a paid plugin, Advanced Sphinx Search Pro by Mirasvit will save you money in the long run.
Purchasing a plugin is a one-off expense. But paying for a high-end server is typically a monthly cost that you will have to incur throughout your Magento 2 instance lifetime.

The installation of the plugin is the usual routine of either copying the files manually or (recommended) installing them via composer.

Let’s review the steps for composer-based installation.

First, let’s install clearmage2 that allows us zero-downtime installation of modules:

sudo yum -y install https://extras.getpagespeed.com/release-latest.rpm
sudo yum install clearmage2

Now, connect to your Magento SSH user, and navigate to the root directory of the Magento 2 store.

If you bought the extension in the mirasvit.com store, copy only the composer config installation instruction from the page My Downloadable Products / View & Download to the SSH console.

composer config ...

Then run the following to install and enable the module:

clearmage2 --install mirasvit/module-search-ultimate:* --enable Mirasvit_Core,Mirasvit_Search,Mirasvit_SearchMysql,Mirasvit_SearchSphinx,Mirasvit_SearchAutocomplete,Mirasvit_Misspell,Mirasvit_SearchLanding,Mirasvit_Report,Mirasvit_SearchReport,Mirasvit_SearchElasticNative

If you bought the extension in the Magento Marketplace, run the command:

clearmage2 --install mirasvit/module-search-ultimate --enable Mirasvit_Core,Mirasvit_Search,Mirasvit_SearchMysql,Mirasvit_SearchSphinx,Mirasvit_SearchAutocomplete,Mirasvit_Misspell,Mirasvit_SearchLanding,Mirasvit_Report,Mirasvit_SearchReport,Mirasvit_SearchElasticNative

Now, reindex search spell-correction index

php -f bin/magento indexer:reindex mst_misspell

Go to System / Search Indexes, configure Product index, and run the reindex for it.

Once the plugin is installed, you can proceed to install the Sphinx search engine software.

Sphinx search engine with libstemmer

Libstemmer controls which characters are accepted as valid and which are not, and how the accepted characters should be transformed (eg. should the case be removed or not).
If the charset table is configured for special language chars, then search phrases “kottkvarn” and “köttkvarn” will return the same result.

As unfortunate as it sometimes is with enterprise OS (full irony intended), system packages of the Sphinx search engine (in EPEL or elsewhere) have a huge oversight of not shipping sphinx with the libstemmer support.

This contributes to my frustrations with much of the RHEL packages from the base repositories.
They are often built with the very minimum functionality and are there just to put a checkmark that they exist. No more, no less.

We need something usable, don’t we? As usual, we bring the packages you love to install and use:

sudo yum -y install https://extras.getpagespeed.com/release-latest.rpm
sudo yum install sphinx

Now there you have it, Sphinx search engine package that doesn’t suck.

It is built with support for Google RE2, libstemmer and everything you need for a bright day :-).

After Sphinx Engine is installed, you need to the Magento search module to it.

Connect with Sphinx Engine

• in Magento admin, go to System -> Search Management -> Settings and proceed to Search Engine Configuration.
• In field Search Engine select External Sphinx Engine option, and fill the following fields.
• Sphinx Host – 127.0.0.1
• Sphinx Port – 9312
• Sphinx Bin Path – /usr/bin/

Now, run full reindex on the command line:

php -f bin/magento mirasvit:search:reindex

Configure search morphology

To configure morphology and charset tables for additional languages, perform the following actions:

• Open file /vendor/mirasvit/module-search-sphinx/src/SearchSphinx/etc/conf/index.conf.
• Configure morphology variable. Standard language codes you can find here. Use two-letter codes from 639-1 column.

For example, to add German to the list of supported languages, you need to set in index.conf:

morphology = stem_enru, libstemmer_de

Read more about morphology here.

• Add charsets to the charset_table variable. List of codes for different charset tables can be found here

For example, to add English and Russian characters, variable should look as shown below:

charset_table = 0..9, A..Z->a..z, _, a..z, \
                U+410..U+42F->U+430..U+44F, U+430..U+44F, U+401->U+451, U+451

Read more about character tables here.

It’s worth noting that you can install the Sphinx search engine on a separate server. But here we covered the most common way which is fine enough for the majority of Magento 2 installations.

With Sphinx search, you have a powerful, language-aware full-text search that is CPU and memory efficient and doesn’t eat your budget every month.