When running updateSearchIndexConfig.php for test.wikidata, the script chokes on the analyzers
Closed, ResolvedPublic
Actions

Description

mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki
content index...
	Fetching Elasticsearch version...1.7.1...ok
	Scanning available plugins...
		langdetect-1.7.0.0-b020e63, wikimedia-extra, analysis-icu, swift-repository, experimental highlighter
		head, whatson
	Infering index identifier...testwikidatawiki_content_1415316749
	Picking analyzer...english
	Index exists so validating...
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...cannot correct
This script encountered an index difference that requires that the index be
copied, indexed to, and then the old index removed. Re-run this script with the
--reindexAndRemoveOk --indexIdentifier=now parameters to do this.

on beta wikidata, I had no problems with this script and locally, normally it's ok.

https://gerrit.wikimedia.org/r/#/c/247788/ is a patch from the Cirrus team that adds an option of updating/validating only the mapping, so could be a work around.

running --reindexAndRemoveOk would probably also be ok in the case of test.wikidata, but if we have the same problem with wikidata then reindex can be more complex and not a great option.

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T33805 Wikipedia Android App 1.1 tracking - current bugs
Open	None	T35704 GeoData should pull coordinates from Wikidata on sites with the Wikibase Client
Resolved	aude	T112531 [Story] see items around me (Special:Nearby)
Duplicate	None	T112866 [Story] Add Wikibase integration with GeoData extension
Resolved	aude	T75482 [Story] Make Wikibase add coordinates to GeoData on wikidata.org
Resolved	aude	T115325 [Task] run refreshLinks on wikidatawiki
Resolved	aude	T115482 [Task] Enable GeoData extension on wikidata.org
Resolved	None	T116185 When running updateSearchIndexConfig.php for test.wikidata, the script chokes on the analyzers

Event Timeline

aude created this task.Oct 21 2015, 3:59 PM

aude raised the priority of this task from to High.

aude updated the task description. (Show Details)

aude added projects: Wikidata, CirrusSearch.

aude subscribed.

Restricted Application added a project: Discovery-ARCHIVED. · View Herald TranscriptOct 21 2015, 3:59 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

aude renamed this task from When running updateSearchIndexCzyonfig.php for test.wikidata, the script chokes on the analyzers to When running updateSearchIndexConfig.php for test.wikidata, the script chokes on the analyzers.Oct 21 2015, 4:00 PM

aude set Security to None.

aude added a parent task: T115482: [Task] Enable GeoData extension on wikidata.org.

From the CirrusSearch README:

B.  Build a copy of the index, reindex to it, and then force a full reindex from source documents.  Uses
more disk space but search should be up the entire time:
 php updateSearchIndexConfig.php --reindexAndRemoveOk --indexIdentifier now
 php forceSearchIndex.php

So this seems to be as expected and should work just fine, unless that readme is wrong.

the patch that adds the --justMapping option got split up into two patches, one that adds the option to the script, and the second patch for "allowing mapping customization with numeric fields"

this is the patch that adds the option:

https://gerrit.wikimedia.org/r/#/c/247861/

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2015-10-27_(1.27.0-wmf.4)).Oct 23 2015, 5:00 PM

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2015-10-13_(1.27.0-wmf.3)).Oct 26 2015, 2:00 PM

backported the --justMapping patch and the problem is resolved now

aude closed this task as Resolved.Oct 26 2015, 4:16 PM

aude added a project: Wikidata-Sprint-2015-10-13.

aude moved this task from Proposed to Done on the Wikidata-Sprint-2015-10-13 board.

• Deskana moved this task from Inbox to Resolved/Invalid/Declined/Legacy on the CirrusSearch board.Dec 31 2015, 5:07 AM