ElasticSearch on MediaWiki - Revision history

Rabbi Bob: /* Job */

2026-08-27T10:33:25Z

Job

← Older revision		Revision as of 10:33, 27 August 2026
Line 158:		Line 158:

	====Job====		====Job====
	# Establish a job that runs every minute		# Establish a job that runs every minute or every 5 minutes or ...
	# crontab -e		# crontab -e
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
Line 164:		Line 164:

	* * * * * /usr/bin/php -d memory_limit=1G /var/www/html/<yourWebDir>/maintenance/run.php runJobs >> /var/log/mediawiki/yourWebSite_runjobs.log 2>&1		* * * * * /usr/bin/php -d memory_limit=1G /var/www/html/<yourWebDir>/maintenance/run.php runJobs >> /var/log/mediawiki/yourWebSite_runjobs.log 2>&1
			/5 * * * /usr/bin/php -d memory_limit=1G /var/www/html/<yourWebDir>/maintenance/run.php runJobs >> /var/log/mediawiki/yourWebSite_runjobs.log 2>&1
	</syntaxhighlight>		</syntaxhighlight>

Rabbi Bob at 10:29, 27 August 2026

2026-08-27T10:29:46Z

← Older revision		Revision as of 10:29, 27 August 2026
Line 7:		Line 7:
	* '''PHP:''' The PHP <code>cURL</code> extension must be enabled.		* '''PHP:''' The PHP <code>cURL</code> extension must be enabled.
	* '''Composer:''' See https://getcomposer.org/install for more information.		* '''Composer:''' See https://getcomposer.org/install for more information.
	* <font color="red">'''DATABASE NAMES MUST BE ALL LOWERCASE'''</font>		* <font color="red">'''DATABASE NAMES MUST BE ALL LOWERCASE!'''</font>

	====Composer Install====		====Composer Install====

Rabbi Bob: /* Step 4: Generate and Bootstrap the Search Index */

2026-08-27T10:26:23Z

Step 4: Generate and Bootstrap the Search Index

← Older revision		Revision as of 10:26, 27 August 2026
Line 86:		Line 86:
	php maintenance/run.php runJobs		php maintenance/run.php runJobs

	#if the above fails out, the db may be too big in various aspects, give the command more memory:		# if the above fails out, the db may be too big in various aspects, give the command more memory:
	php -d memory_limit=1G maintenance/run.php runJobs		php -d memory_limit=1G maintenance/run.php runJobs
	</syntaxhighlight>		</syntaxhighlight>

			# Verify the indexes, this should show your db with count details. If 0, re-run the above commands again and re-test.
			curl -X GET "localhost:9200/_cat/indices?v"


	*: ''Note: For smaller wikis, running just <code>php maintenance/run.php CirrusSearch:ForceSearchIndex</code> is usually sufficient.''		*: ''Note: For smaller wikis, running just <code>php maintenance/run.php CirrusSearch:ForceSearchIndex</code> is usually sufficient.''

Rabbi Bob: /* Step 6: LocalSettings.php Cleanup */

2026-08-27T10:18:12Z

Step 6: LocalSettings.php Cleanup

← Older revision		Revision as of 10:18, 27 August 2026
Line 107:		Line 107:
	## CUSTOM ELASTIC SEARCH		## CUSTOM ELASTIC SEARCH
	// Force CirrusSearch overrides at the very end of LocalSettings.php		// Force CirrusSearch overrides at the very end of LocalSettings.php
			# https://gerrit.wikimedia.org/g/mediawiki/extensions/CirrusSearch/%2B/HEAD/docs/settings.txt
	wfLoadExtension( 'Elastica' );		wfLoadExtension( 'Elastica' );
	wfLoadExtension( 'CirrusSearch' );		wfLoadExtension( 'CirrusSearch' );
Line 113:		Line 114:
	$wgCirrusSearchServers = [ '127.0.0.1' ];		$wgCirrusSearchServers = [ '127.0.0.1' ];
	$wgSearchType = 'CirrusSearch';		$wgSearchType = 'CirrusSearch';
	~~$wgCirrusSearchEnableDynamicHighlighter = false;~~

Rabbi Bob: /* Step 4: Generate and Bootstrap the Search Index */

2026-08-27T10:11:50Z

Step 4: Generate and Bootstrap the Search Index

← Older revision		Revision as of 10:11, 27 August 2026
Line 81:		Line 81:

	# '''Populate the index with your wiki's page content:'''		# '''Populate the index with your wiki's page content:'''
	<syntaxhighlight lang="~~bash~~">		<syntaxhighlight lang="php">
	php maintenance/run.php CirrusSearch:ForceSearchIndex --skipLinks --indexOnSkip		php maintenance/run.php CirrusSearch:ForceSearchIndex --skipLinks --indexOnSkip
	php maintenance/run.php CirrusSearch:ForceSearchIndex --skipParse		php maintenance/run.php CirrusSearch:ForceSearchIndex --skipParse

Rabbi Bob: /* Step 7: Setup Crontab Job */

2026-08-27T10:11:08Z

Step 7: Setup Crontab Job

← Older revision		Revision as of 10:11, 27 August 2026
Line 127:		Line 127:

	=== Step 7: Setup Crontab Job ===		=== Step 7: Setup Crontab Job ===
			====Logs====
			# Setup a log dir (probably should revisit this some day and do a proper rotation)
			<syntaxhighlight lang="bash">
			sudo mkdir -p /var/log/mediawiki
			sudo chown it:it /var/log/mediawiki
			</syntaxhighlight>

			====Log Rotate====
			<syntaxhighlight lang="bash">
			sudo nano /etc/logrotate.d/mediawiki
			</syntaxhighlight>

			<syntaxhighlight lang="bash">
			/var/log/mediawiki/*.log {
			daily
			missingok
			rotate 14
			compress
			delaycompress
			notifempty
			create 640 www-data www-data
			}
			</syntaxhighlight>


	====Job====		====Job====
	# Establish a job that runs every minute		# Establish a job that runs every minute
	# crontab -e		# crontab -e
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	* * * * * /usr/bin/php /var/www/html/<yourWebDir>/maintenance/run.php runJobs ~~--maxtime=60~~ >> /var/log/mediawiki/~~runJobs~~.log 2>&1		sudo -u www-data crontab -e

			* * * * * /usr/bin/php -d memory_limit=1G /var/www/html/<yourWebDir>/maintenance/run.php runJobs >> /var/log/mediawiki/yourWebSite_runjobs.log 2>&1
	</syntaxhighlight>		</syntaxhighlight>
	====~~Logs~~====
	# ~~Setup a log dir (probably should revisit this some day and do a proper rotation)~~		====Test====
			# Wait at least 1 minute before doing the checking
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	sudo ~~mkdir~~ -p /var/log/mediawiki		sudo tail -f /var/log/mediawiki/YourWebSite_runjobs.log
	~~sudo chown it:it /var~~/log~~/mediawiki~~
	</syntaxhighlight>		</syntaxhighlight>

Rabbi Bob: /* Step 6: LocalSettings.php Cleanup */

2026-08-27T10:03:28Z

Step 6: LocalSettings.php Cleanup

← Older revision		Revision as of 10:03, 27 August 2026
Line 110:		Line 110:
	wfLoadExtension( 'CirrusSearch' );		wfLoadExtension( 'CirrusSearch' );

			// Ensure fallback to core MySQL is explicitly disabled for testing
			$wgCirrusSearchServers = [ '127.0.0.1' ];
	$wgSearchType = 'CirrusSearch';		$wgSearchType = 'CirrusSearch';
	$~~wgCirrusSearchServers~~ = ~~[ '127.0.0.1' ]~~;		$wgCirrusSearchEnableDynamicHighlighter = false;


	// Force exact database prefix matching for CirrusSearch		// Force exact database prefix matching for CirrusSearch
Line 119:		Line 122:
	]		]
	];		];

	~~// Ensure fallback to core MySQL is explicitly disabled for testing~~
	~~$wgSearchType = 'CirrusSearch';~~
	~~$wgCirrusSearchEnableDynamicHighlighter = false;~~

	$wgJobRunRate = 0;		$wgJobRunRate = 0;

Rabbi Bob: /* Step 4: Generate and Bootstrap the Search Index */

2026-08-27T10:00:32Z

Step 4: Generate and Bootstrap the Search Index

← Older revision		Revision as of 10:00, 27 August 2026
Line 84:		Line 84:
	php maintenance/run.php CirrusSearch:ForceSearchIndex --skipLinks --indexOnSkip		php maintenance/run.php CirrusSearch:ForceSearchIndex --skipLinks --indexOnSkip
	php maintenance/run.php CirrusSearch:ForceSearchIndex --skipParse		php maintenance/run.php CirrusSearch:ForceSearchIndex --skipParse
			php maintenance/run.php runJobs

			#if the above fails out, the db may be too big in various aspects, give the command more memory:
			php -d memory_limit=1G maintenance/run.php runJobs
	</syntaxhighlight>		</syntaxhighlight>

Rabbi Bob: /* Prerequisites */

2026-08-26T08:56:25Z

Prerequisites

← Older revision		Revision as of 08:56, 26 August 2026
Line 7:		Line 7:
	* '''PHP:''' The PHP <code>cURL</code> extension must be enabled.		* '''PHP:''' The PHP <code>cURL</code> extension must be enabled.
	* '''Composer:''' See https://getcomposer.org/install for more information.		* '''Composer:''' See https://getcomposer.org/install for more information.
			* <font color="red">'''DATABASE NAMES MUST BE ALL LOWERCASE'''</font>

	====Composer Install====		====Composer Install====

Rabbi Bob at 08:54, 26 August 2026

2026-08-26T08:54:31Z

← Older revision		Revision as of 08:54, 26 August 2026
Line 6:		Line 6:
	* '''MediaWiki 1.39+:''' Recommended Elasticsearch 7.10.2 (or OpenSearch 1.3+).		* '''MediaWiki 1.39+:''' Recommended Elasticsearch 7.10.2 (or OpenSearch 1.3+).
	* '''PHP:''' The PHP <code>cURL</code> extension must be enabled.		* '''PHP:''' The PHP <code>cURL</code> extension must be enabled.
			* '''Composer:''' See https://getcomposer.org/install for more information.

			====Composer Install====
			<syntaxhighlight lang="bash">
			sudo apt update
			sudo apt upgrade
			sudo apt install curl php-cli php-mbstring git unzip
			php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
			php composer-setup.php --install-dir=/usr/local/bin --filename=composer
			</syntaxhighlight>

	=== Step 1: Install & Start Elasticsearch ===		=== Step 1: Install & Start Elasticsearch ===
Line 40:		Line 50:
	# '''Install PHP Dependencies:''' If you cloned the extensions via Git, you must install the Composer dependencies inside both extension folders:		# '''Install PHP Dependencies:''' If you cloned the extensions via Git, you must install the Composer dependencies inside both extension folders:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	cd Elastica && composer install --no-dev && cd ..		cd Elastica && composer config policy.advisories.block false && composer install --no-dev && cd ..
	cd CirrusSearch && composer install --no-dev && cd ..		cd CirrusSearch && composer config policy.advisories.block false && composer install --no-dev && cd ..
	</syntaxhighlight>		</syntaxhighlight>