Page MenuHomePhabricator
Create Task
Maniphest T255566

[LEX] [TECH] Figure out why LexemeEditEntityTest::testGivenInvalidDataInClearRequest_errorIsReported is throwing an Exception/logging an error with AbuseFilter enabled
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

This is not only a CI error, but also a production error. See T255566#9134792 for the latest occurrence.

In an effort to resolve errors ending up in mw-error.log during a CI run in T249838: Fix WikibaseLexeme UnsupportedObjectExceptions in Wikibase CI builds, it was discovered that the tests LexemeEditEntityTest::testGivenInvalidDataInClearRequest_errorIsReported thows an UnsupportedObjectException if AbuseFilter is enabled that ends up in that file.

The exception logging for that test was temporarily disabled to clear the way for asserting that mw-error.log is empty during a CI run. However, we should still figure out why this happens in the first place and fix it properly.

Details

SubjectRepoBranchLines /-
Revert "Disable exception logger for a test"mediawiki/extensions/WikibaseLexememaster 0 -3
EntityContentDataCodec: Don't log exceptionsmediawiki/extensions/Wikibasemaster 0 -2
phpunit: Remove setNullLoggermediawiki/extensions/WikibaseLexememaster 0 -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

Michael created this task.Jun 16 2020, 2:27 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptJun 16 2020, 2:27 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Addshore subscribed.Jun 16 2020, 3:27 PM
Addshore moved this task from Inbox to Legacy Inbox on the [DEPRECATED] wdwb-tech board.Mar 17 2021, 5:19 PM
Addshore moved this task from Legacy Inbox to To Prioritize on the [DEPRECATED] wdwb-tech board.Apr 20 2021, 7:13 AM

This was reported a year ago but still open, so moving to the backlog to be prioritized

Lucas_Werkmeister_WMDE added a comment.EditedJun 15 2021, 10:26 AM

This can also happen in production logstash, by the way:

Error
normalized_message
[{reqId}] {exception_url}   Serializers\Exceptions\UnsupportedObjectException: Can not serialize incomplete Lexeme
exception.trace
from /srv/mediawiki/php-1.37.0-wmf.9/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(109)
#0 /srv/mediawiki/php-1.37.0-wmf.9/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(80): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->getSerialized(Wikibase\Lexeme\Domain\Model\Lexeme)
#1 /srv/mediawiki/php-1.37.0-wmf.9/vendor/serialization/serialization/src/Serializers/DispatchingSerializer.php(42): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#2 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/lib/includes/Store/EntityContentDataCodec.php(152): Serializers\DispatchingSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#3 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/Content/EntityContent.php(327): Wikibase\Lib\Store\EntityContentDataCodec->encodeEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string)
#4 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/RepoHooks.php(648): Wikibase\Repo\Content\EntityContent->getTextForFilters()
#5 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(338): Wikibase\Repo\RepoHooks::onAbuseFilterContentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, NULL)
#6 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#7 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/Hooks/AbuseFilterHookRunner.php(89): MediaWiki\HookContainer\HookContainer->run(string, array)
#8 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/TextExtractor.php(85): MediaWiki\Extension\AbuseFilter\Hooks\AbuseFilterHookRunner->onAbuseFilter_contentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, NULL)
#9 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/AbuseFilterHooks.php(76): MediaWiki\Extension\AbuseFilter\TextExtractor->contentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent)
#10 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/AbuseFilterHooks.php(45): MediaWiki\Extension\AbuseFilter\AbuseFilterHooks::filterEdit(DerivativeContext, User, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, string, string)
#11 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(338): MediaWiki\Extension\AbuseFilter\AbuseFilterHooks::onEditFilterMergedContent(DerivativeContext, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, Status, string, User, boolean, string)
#12 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#13 /srv/mediawiki/php-1.37.0-wmf.9/includes/Hooks.php(137): MediaWiki\HookContainer\HookContainer->run(string, array, array)
#14 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/MediawikiEditFilterHookRunner.php(105): Hooks::run(string, array)
#15 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/StatsdTimeRecordingEditFilterHookRunner.php(47): Wikibase\Repo\EditEntity\MediawikiEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#16 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/MediawikiEditEntity.php(723): Wikibase\Repo\EditEntity\StatsdTimeRecordingEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#17 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/StatsdSaveTimeRecordingEditEntity.php(77): Wikibase\Repo\EditEntity\MediawikiEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, boolean, array)
#18 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/Api/EntitySavingHelper.php(392): Wikibase\Repo\EditEntity\StatsdSaveTimeRecordingEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, NULL, array)
#19 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/Api/ModifyEntity.php(327): Wikibase\Repo\Api\EntitySavingHelper->attemptSaveEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string, array, DerivativeContext)
#20 /srv/mediawiki/php-1.37.0-wmf.9/includes/api/ApiMain.php(1714): Wikibase\Repo\Api\ModifyEntity->execute()
#21 /srv/mediawiki/php-1.37.0-wmf.9/includes/api/ApiMain.php(684): ApiMain->executeAction()
#22 /srv/mediawiki/php-1.37.0-wmf.9/includes/api/ApiMain.php(655): ApiMain->executeActionWithErrorHandling()
#23 /srv/mediawiki/php-1.37.0-wmf.9/api.php(90): ApiMain->execute()
#24 /srv/mediawiki/php-1.37.0-wmf.9/api.php(45): wfApiMain()
#25 /srv/mediawiki/w/api.php(3): require(string)
#26 {main}
exception.previous.trace
from /srv/mediawiki/php-1.37.0-wmf.9/extensions/WikibaseLexeme/src/Domain/Model/Lexeme.php(232)
#0 /srv/mediawiki/php-1.37.0-wmf.9/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(106): Wikibase\Lexeme\Domain\Model\Lexeme->getLexicalCategory()
#1 /srv/mediawiki/php-1.37.0-wmf.9/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(80): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->getSerialized(Wikibase\Lexeme\Domain\Model\Lexeme)
#2 /srv/mediawiki/php-1.37.0-wmf.9/vendor/serialization/serialization/src/Serializers/DispatchingSerializer.php(42): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#3 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/lib/includes/Store/EntityContentDataCodec.php(152): Serializers\DispatchingSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#4 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/Content/EntityContent.php(327): Wikibase\Lib\Store\EntityContentDataCodec->encodeEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string)
#5 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/RepoHooks.php(648): Wikibase\Repo\Content\EntityContent->getTextForFilters()
#6 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(338): Wikibase\Repo\RepoHooks::onAbuseFilterContentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, NULL)
#7 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#8 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/Hooks/AbuseFilterHookRunner.php(89): MediaWiki\HookContainer\HookContainer->run(string, array)
#9 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/TextExtractor.php(85): MediaWiki\Extension\AbuseFilter\Hooks\AbuseFilterHookRunner->onAbuseFilter_contentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, NULL)
#10 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/AbuseFilterHooks.php(76): MediaWiki\Extension\AbuseFilter\TextExtractor->contentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent)
#11 /srv/mediawiki/php-1.37.0-wmf.9/extensions/AbuseFilter/includes/AbuseFilterHooks.php(45): MediaWiki\Extension\AbuseFilter\AbuseFilterHooks::filterEdit(DerivativeContext, User, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, string, string)
#12 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(338): MediaWiki\Extension\AbuseFilter\AbuseFilterHooks::onEditFilterMergedContent(DerivativeContext, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, Status, string, User, boolean, string)
#13 /srv/mediawiki/php-1.37.0-wmf.9/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#14 /srv/mediawiki/php-1.37.0-wmf.9/includes/Hooks.php(137): MediaWiki\HookContainer\HookContainer->run(string, array, array)
#15 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/MediawikiEditFilterHookRunner.php(105): Hooks::run(string, array)
#16 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/StatsdTimeRecordingEditFilterHookRunner.php(47): Wikibase\Repo\EditEntity\MediawikiEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#17 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/MediawikiEditEntity.php(723): Wikibase\Repo\EditEntity\StatsdTimeRecordingEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#18 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/EditEntity/StatsdSaveTimeRecordingEditEntity.php(77): Wikibase\Repo\EditEntity\MediawikiEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, boolean, array)
#19 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/Api/EntitySavingHelper.php(392): Wikibase\Repo\EditEntity\StatsdSaveTimeRecordingEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, NULL, array)
#20 /srv/mediawiki/php-1.37.0-wmf.9/extensions/Wikibase/repo/includes/Api/ModifyEntity.php(327): Wikibase\Repo\Api\EntitySavingHelper->attemptSaveEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string, array, DerivativeContext)
#21 /srv/mediawiki/php-1.37.0-wmf.9/includes/api/ApiMain.php(1714): Wikibase\Repo\Api\ModifyEntity->execute()
#22 /srv/mediawiki/php-1.37.0-wmf.9/includes/api/ApiMain.php(684): ApiMain->executeAction()
#23 /srv/mediawiki/php-1.37.0-wmf.9/includes/api/ApiMain.php(655): ApiMain->executeActionWithErrorHandling()
#24 /srv/mediawiki/php-1.37.0-wmf.9/api.php(90): ApiMain->execute()
#25 /srv/mediawiki/php-1.37.0-wmf.9/api.php(45): wfApiMain()
#26 /srv/mediawiki/w/api.php(3): require(string)
#27 {main}

I assume the reason we haven’t spotted it as a production error before is that these broken requests are almost never made. (The API response is not an internal error, by the way, but a regular API usage error. This exception is only logged internally.)

Addshore moved this task from To Prioritize to Triaged Low (0-50) on the [DEPRECATED] wdwb-tech board.Jun 15 2021, 11:27 AM
Addshore added projects: Wikimedia-production-error, Wikidata Lexicographical data.
Krinkle moved this task from Untriaged to June 2021 on the Wikimedia-production-error board.Jul 13 2021, 1:01 AM
Addshore triaged this task as Low priority.Jul 14 2021, 11:12 AM
kostajh subscribed.Jan 18 2022, 9:23 PM

I just saw this in a Selenium test context (logs https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-selenium-docker/97086/console#console-section-12):

21:55:30 [0-11] RUNNING in chrome - /tests/selenium/specs/statement.add.js
21:55:31 [0-11] 2022-01-18T20:55:38.000Z ERROR @wdio/sync: Evaluation failed: TypeError: mw.loader.using is not a function
21:55:38 [0-11]     at eval (eval at <anonymous> (:20:16), <anonymous>:5:14)
21:55:51 [0-11]     at eval (eval at <anonymous> (:20:16), <anonymous>:11:6)
21:55:51 [0-11]     at __puppeteer_evaluation_script__:20:16
21:55:51 [0-11]     at __puppeteer_evaluation_script__:2:12
21:55:51 [0-11] Error: Evaluation failed: TypeError: mw.loader.using is not a function
21:55:51 [0-11]     at ExecutionContext._evaluateInternal (/workspace/src/extensions/WikibaseLexeme/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
21:55:51 [0-11]     at processTicksAndRejections (internal/process/task_queues.js:95:5)
21:55:51 [0-11]     at async ExecutionContext.evaluate (/workspace/src/extensions/WikibaseLexeme/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:110:16)
21:55:51 [0-11]     at async ElementHandle.evaluate (/workspace/src/extensions/WikibaseLexeme/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:103:16)
21:55:51 [0-11]     at async ElementHandle.$eval (/workspace/src/extensions/WikibaseLexeme/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:806:24)
21:55:51 [0-11]     at async DevToolsDriver.executeAsyncScript (/workspace/src/extensions/WikibaseLexeme/node_modules/devtools/build/commands/executeAsyncScript.js:31:20)
21:55:51 [0-11]     at async Browser.wrappedCommand (/workspace/src/extensions/WikibaseLexeme/node_modules/devtools/build/devtoolsdriver.js:102:26)
21:55:51 [0-11]     at Browser.<anonymous> (/workspace/src/extensions/WikibaseLexeme/node_modules/@wdio/utils/build/shim.js:270:24)
21:55:51 [0-11]     at Object.beforeSuite (/workspace/src/extensions/WikibaseLexeme/tests/selenium/wdio.conf.js:41:11)
21:55:51 [0-11] 2: starting lexeme creation
21:55:51 [0-11] 3: have categoryValue
21:55:51 [0-11] 3: have languageValue
21:55:51 [0-11] Trace: WARNING: LexemeApi not initialized
21:55:51 [0-11]     at LexemeApi.getBot (/workspace/src/extensions/WikibaseLexeme/tests/selenium/lexeme.api.js:44:12)
21:55:51 [0-11]     at /workspace/src/extensions/WikibaseLexeme/tests/selenium/lexeme.api.js:81:16
21:55:51 [0-11]     at processTicksAndRejections (internal/process/task_queues.js:95:5)
21:55:51 [0-11]     at async Browser.runCommandWithHooks (/workspace/src/extensions/WikibaseLexeme/node_modules/@wdio/sync/build/wrapCommand.js:105:25)
21:55:51 [0-11] 585: have bot
21:55:51 [0-11] Error in "Lexeme:Statements.can be added"
21:55:51 Error: failed-save: The save has failed.
21:55:51     at /workspace/src/extensions/WikibaseLexeme/node_modules/mwbot/src/index.js:262:31
21:55:51     at tryCatcher (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/util.js:16:23)
21:55:51     at Promise._settlePromiseFromHandler (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/promise.js:547:31)
21:55:51     at Promise._settlePromise (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/promise.js:604:18)
21:55:51     at Promise._settlePromise0 (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/promise.js:649:10)
21:55:51     at Promise._settlePromises (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/promise.js:729:18)
21:55:51     at _drainQueueStep (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/async.js:93:12)
21:55:51     at _drainQueue (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/async.js:86:9)
21:55:51     at Async._drainQueues (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/async.js:102:5)
21:55:51     at Immediate.Async.drainQueues [as _onImmediate] (/workspace/src/extensions/WikibaseLexeme/node_modules/bluebird/js/release/async.js:15:14)
21:55:51     at processImmediate (internal/timers.js:464:21)
21:55:51     at Browser.<anonymous> (/workspace/src/extensions/WikibaseLexeme/node_modules/@wdio/utils/build/shim.js:270:24)
21:55:51     at Context.<anonymous> (/workspace/src/extensions/WikibaseLexeme/tests/selenium/specs/statement.add.js:11:28)
21:55:51 [0-11] RETRYING in chrome - /tests/selenium/specs/statement.add.js
21:55:51 [0-11] RUNNING in chrome - /tests/selenium/specs/statement.add.js
21:55:51 [0-11] 1642539344266: Waiting 0 ms for replication lag of 0 s
21:55:51 [0-11] 1: starting lexeme creation
21:55:51 [0-11] 48: have categoryValue
21:55:51 [0-11] 75: have languageValue
21:55:51 [0-11] 76: have bot
21:55:51 [0-11] 175: Lexeme created
21:55:51 [0-11] PASSED in chrome - /tests/selenium/specs/statement.add.js (1 retries)

The retry attempt succeeded. But unfortunately the exception is in mw-error.log so the build fails.

The logged exception:

1:57:51 mw-error.log:2022-01-18 20:55:38 33b9a2b24d90 wikidb: [24970b2389fc680449911cee] /api.php?format=json   Serializers\Exceptions\UnsupportedObjectException: Can not serialize incomplete Lexeme
21:57:51 mw-error.log:#0 /workspace/src/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(80): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->getSerialized(Wikibase\Lexeme\Domain\Model\Lexeme)
21:57:51 mw-error.log:#1 /workspace/src/vendor/serialization/serialization/src/Serializers/DispatchingSerializer.php(42): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
21:57:51 mw-error.log:#2 /workspace/src/extensions/Wikibase/lib/includes/Store/EntityContentDataCodec.php(152): Serializers\DispatchingSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
21:57:51 mw-error.log:#3 /workspace/src/extensions/Wikibase/repo/includes/Content/EntityContent.php(185): Wikibase\Lib\Store\EntityContentDataCodec->encodeEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string)
21:57:51 mw-error.log:#4 /workspace/src/extensions/Wikibase/repo/includes/RepoHooks.php(693): Wikibase\Repo\Content\EntityContent->getTextForFilters()
21:57:51 mw-error.log:#5 /workspace/src/includes/HookContainer/HookContainer.php(338): Wikibase\Repo\RepoHooks::onAbuseFilterContentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, NULL)
21:57:51 mw-error.log:#6 /workspace/src/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
21:57:51 mw-error.log:#7 /workspace/src/extensions/AbuseFilter/includes/Hooks/AbuseFilterHookRunner.php(89): MediaWiki\HookContainer\HookContainer->run(string, array)
21:57:51 mw-error.log:#8 /workspace/src/extensions/AbuseFilter/includes/TextExtractor.php(85): MediaWiki\Extension\AbuseFilter\Hooks\AbuseFilterHookRunner->onAbuseFilter_contentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, NULL)
21:57:51 mw-error.log:#9 /workspace/src/extensions/AbuseFilter/includes/VariableGenerator/RunVariableGenerator.php(193): MediaWiki\Extension\AbuseFilter\TextExtractor->contentToString(Wikibase\Lexeme\MediaWiki\Content\LexemeContent)
21:57:51 mw-error.log:#10 /workspace/src/extensions/AbuseFilter/includes/Hooks/Handlers/FilteredActionsHandler.php(136): MediaWiki\Extension\AbuseFilter\VariableGenerator\RunVariableGenerator->getEditVars(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, string, string, WikiPage)
21:57:51 mw-error.log:#11 /workspace/src/extensions/AbuseFilter/includes/Hooks/Handlers/FilteredActionsHandler.php(88): MediaWiki\Extension\AbuseFilter\Hooks\Handlers\FilteredActionsHandler->filterEdit(DerivativeContext, User, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, string, string)
21:57:51 mw-error.log:#12 /workspace/src/includes/HookContainer/HookContainer.php(160): MediaWiki\Extension\AbuseFilter\Hooks\Handlers\FilteredActionsHandler->onEditFilterMergedContent(DerivativeContext, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, Status, string, User, boolean, string)
21:57:51 mw-error.log:#13 /workspace/src/includes/Hooks.php(137): MediaWiki\HookContainer\HookContainer->run(string, array, array)
21:57:51 mw-error.log:#14 /workspace/src/extensions/Wikibase/repo/includes/EditEntity/MediawikiEditFilterHookRunner.php(110): Hooks::run(string, array)
21:57:51 mw-error.log:#15 /workspace/src/extensions/Wikibase/repo/includes/EditEntity/StatsdTimeRecordingEditFilterHookRunner.php(47): Wikibase\Repo\EditEntity\MediawikiEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
21:57:51 mw-error.log:#16 /workspace/src/extensions/Wikibase/repo/includes/EditEntity/MediawikiEditEntity.php(727): Wikibase\Repo\EditEntity\StatsdTimeRecordingEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
21:57:51 mw-error.log:#17 /workspace/src/extensions/Wikibase/repo/includes/EditEntity/StatsdSaveTimeRecordingEditEntity.php(77): Wikibase\Repo\EditEntity\MediawikiEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, boolean, array)
21:57:51 mw-error.log:#18 /workspace/src/extensions/Wikibase/repo/includes/Api/EntitySavingHelper.php(392): Wikibase\Repo\EditEntity\StatsdSaveTimeRecordingEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, NULL, array)
21:57:51 mw-error.log:#19 /workspace/src/extensions/Wikibase/repo/includes/Api/ModifyEntity.php(339): Wikibase\Repo\Api\EntitySavingHelper->attemptSaveEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string, array, DerivativeContext)
21:57:51 mw-error.log:#20 /workspace/src/includes/api/ApiMain.php(1889): Wikibase\Repo\Api\ModifyEntity->execute()
21:57:51 mw-error.log:#21 /workspace/src/includes/api/ApiMain.php(868): ApiMain->executeAction()
21:57:51 mw-error.log:#22 /workspace/src/includes/api/ApiMain.php(839): ApiMain->executeActionWithErrorHandling()
21:57:51 mw-error.log:#23 /workspace/src/api.php(90): ApiMain->execute()
21:57:51 mw-error.log:#24 /workspace/src/api.php(45): wfApiMain()
21:57:51 mw-error.log:#25 {main}
kostajh added a project: ci-test-error.Feb 7 2022, 8:47 AM

I'm not 100% sure, but it seems like this occurs more frequently in CI after T285649: Switch all Quibble Selenium and api-testing jobs to use apache was done.

kostajh added a parent task: T285649: Switch all Quibble Selenium and api-testing jobs to use apache.Feb 7 2022, 8:47 AM
kostajh added projects: AbuseFilter, SpamBlacklist.Feb 7 2022, 10:31 AM

It looks like the errors discussed in the comments are a little different from what is in the description; looking in Logstash I found that the exception is Can not access uninitialized field, and the backtrace includes SpamBlacklist extensions. I also only see two instances of it in the last 9 months.

kostajh added a parent task: T299492: Quibble ci-fullrun jobs should use Apache backend.Feb 7 2022, 10:55 AM
gerritbot added a comment.Jul 12 2022, 4:13 PM

Change 813265 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikibaseLexeme@master] phpunit: Remove setNullLogger

https://gerrit.wikimedia.org/r/813265

gerritbot added a project: Patch-For-Review.Jul 12 2022, 4:13 PM
gerritbot added a comment.Jul 12 2022, 4:38 PM

Change 813265 abandoned by Kosta Harlan:

[mediawiki/extensions/WikibaseLexeme@master] phpunit: Remove setNullLogger

Reason:

https://gerrit.wikimedia.org/r/813265

Maintenance_bot removed a project: Patch-For-Review.Jul 12 2022, 5:31 PM
Michael added a project: Browser-Tests.Aug 24 2022, 11:56 AM
Addshore unsubscribed.Jun 27 2023, 12:43 PM
thcipriani subscribed.Aug 31 2023, 3:58 PM

Still happens in production (once in the last 90 days). Fresh info:

Error
normalized_message
[{reqId}] {exception_url}   Serializers\Exceptions\UnsupportedObjectException: Can not serialize incomplete Lexeme
exception.trace
from /srv/mediawiki/php-1.41.0-wmf.22/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(109)
#0 /srv/mediawiki/php-1.41.0-wmf.22/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(80): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->getSerialized(Wikibase\Lexeme\Domain\Model\Lexeme)
#1 /srv/mediawiki/php-1.41.0-wmf.22/vendor/serialization/serialization/src/Serializers/DispatchingSerializer.php(42): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#2 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/lib/includes/Store/EntityContentDataCodec.php(152): Serializers\DispatchingSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#3 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/Content/EntityHandler.php(381): Wikibase\Lib\Store\EntityContentDataCodec->encodeEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string)
#4 /srv/mediawiki/php-1.41.0-wmf.22/includes/content/AbstractContent.php(177): Wikibase\Repo\Content\EntityHandler->serializeContent(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, string)
#5 /srv/mediawiki/php-1.41.0-wmf.22/includes/Storage/PageEditStash.php(437): AbstractContent->serialize(string)
#6 /srv/mediawiki/php-1.41.0-wmf.22/includes/Storage/PageEditStash.php(264): MediaWiki\Storage\PageEditStash->getContentHash(Wikibase\Lexeme\MediaWiki\Content\LexemeContent)
#7 /srv/mediawiki/php-1.41.0-wmf.22/extensions/SpamBlacklist/includes/Hooks.php(108): MediaWiki\Storage\PageEditStash->checkCache(MediaWiki\Title\Title, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, User)
#8 /srv/mediawiki/php-1.41.0-wmf.22/includes/HookContainer/HookContainer.php(160): MediaWiki\Extension\SpamBlacklist\Hooks->onEditFilterMergedContent(DerivativeContext, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, Status, string, User, boolean, string)
#9 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/MediaWikiEditFilterHookRunner.php(122): MediaWiki\HookContainer\HookContainer->run(string, array)
#10 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/StatsdTimeRecordingEditFilterHookRunner.php(48): Wikibase\Repo\EditEntity\MediaWikiEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#11 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/MediaWikiEditEntity.php(738): Wikibase\Repo\EditEntity\StatsdTimeRecordingEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#12 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/StatsdSaveTimeRecordingEditEntity.php(77): Wikibase\Repo\EditEntity\MediaWikiEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, boolean, array)
#13 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/Api/EntitySavingHelper.php(391): Wikibase\Repo\EditEntity\StatsdSaveTimeRecordingEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, NULL, array)
#14 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/Api/ModifyEntity.php(340): Wikibase\Repo\Api\EntitySavingHelper->attemptSaveEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string, array, DerivativeContext)
#15 /srv/mediawiki/php-1.41.0-wmf.22/includes/api/ApiMain.php(1921): Wikibase\Repo\Api\ModifyEntity->execute()
#16 /srv/mediawiki/php-1.41.0-wmf.22/includes/api/ApiMain.php(898): ApiMain->executeAction()
#17 /srv/mediawiki/php-1.41.0-wmf.22/includes/api/ApiMain.php(869): ApiMain->executeActionWithErrorHandling()
#18 /srv/mediawiki/php-1.41.0-wmf.22/api.php(95): ApiMain->execute()
#19 /srv/mediawiki/php-1.41.0-wmf.22/api.php(48): wfApiMain()
#20 /srv/mediawiki/w/api.php(3): require(string)
#21 {main}
exception.previous.trace
from /srv/mediawiki/php-1.41.0-wmf.22/extensions/WikibaseLexeme/src/Domain/Model/Lexeme.php(232)
#0 /srv/mediawiki/php-1.41.0-wmf.22/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(106): Wikibase\Lexeme\Domain\Model\Lexeme->getLexicalCategory()
#1 /srv/mediawiki/php-1.41.0-wmf.22/extensions/WikibaseLexeme/src/Serialization/StorageLexemeSerializer.php(80): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->getSerialized(Wikibase\Lexeme\Domain\Model\Lexeme)
#2 /srv/mediawiki/php-1.41.0-wmf.22/vendor/serialization/serialization/src/Serializers/DispatchingSerializer.php(42): Wikibase\Lexeme\Serialization\StorageLexemeSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#3 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/lib/includes/Store/EntityContentDataCodec.php(152): Serializers\DispatchingSerializer->serialize(Wikibase\Lexeme\Domain\Model\Lexeme)
#4 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/Content/EntityHandler.php(381): Wikibase\Lib\Store\EntityContentDataCodec->encodeEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string)
#5 /srv/mediawiki/php-1.41.0-wmf.22/includes/content/AbstractContent.php(177): Wikibase\Repo\Content\EntityHandler->serializeContent(Wikibase\Lexeme\MediaWiki\Content\LexemeContent, string)
#6 /srv/mediawiki/php-1.41.0-wmf.22/includes/Storage/PageEditStash.php(437): AbstractContent->serialize(string)
#7 /srv/mediawiki/php-1.41.0-wmf.22/includes/Storage/PageEditStash.php(264): MediaWiki\Storage\PageEditStash->getContentHash(Wikibase\Lexeme\MediaWiki\Content\LexemeContent)
#8 /srv/mediawiki/php-1.41.0-wmf.22/extensions/SpamBlacklist/includes/Hooks.php(108): MediaWiki\Storage\PageEditStash->checkCache(MediaWiki\Title\Title, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, User)
#9 /srv/mediawiki/php-1.41.0-wmf.22/includes/HookContainer/HookContainer.php(160): MediaWiki\Extension\SpamBlacklist\Hooks->onEditFilterMergedContent(DerivativeContext, Wikibase\Lexeme\MediaWiki\Content\LexemeContent, Status, string, User, boolean, string)
#10 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/MediaWikiEditFilterHookRunner.php(122): MediaWiki\HookContainer\HookContainer->run(string, array)
#11 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/StatsdTimeRecordingEditFilterHookRunner.php(48): Wikibase\Repo\EditEntity\MediaWikiEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#12 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/MediaWikiEditEntity.php(738): Wikibase\Repo\EditEntity\StatsdTimeRecordingEditFilterHookRunner->run(Wikibase\Lexeme\Domain\Model\Lexeme, DerivativeContext, string)
#13 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/EditEntity/StatsdSaveTimeRecordingEditEntity.php(77): Wikibase\Repo\EditEntity\MediaWikiEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, boolean, array)
#14 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/Api/EntitySavingHelper.php(391): Wikibase\Repo\EditEntity\StatsdSaveTimeRecordingEditEntity->attemptSave(Wikibase\Lexeme\Domain\Model\Lexeme, string, integer, string, NULL, array)
#15 /srv/mediawiki/php-1.41.0-wmf.22/extensions/Wikibase/repo/includes/Api/ModifyEntity.php(340): Wikibase\Repo\Api\EntitySavingHelper->attemptSaveEntity(Wikibase\Lexeme\Domain\Model\Lexeme, string, array, DerivativeContext)
#16 /srv/mediawiki/php-1.41.0-wmf.22/includes/api/ApiMain.php(1921): Wikibase\Repo\Api\ModifyEntity->execute()
#17 /srv/mediawiki/php-1.41.0-wmf.22/includes/api/ApiMain.php(898): ApiMain->executeAction()
#18 /srv/mediawiki/php-1.41.0-wmf.22/includes/api/ApiMain.php(869): ApiMain->executeActionWithErrorHandling()
#19 /srv/mediawiki/php-1.41.0-wmf.22/api.php(95): ApiMain->execute()
#20 /srv/mediawiki/php-1.41.0-wmf.22/api.php(48): wfApiMain()
#21 /srv/mediawiki/w/api.php(3): require(string)
#22 {main}
Michael changed the subtype of this task from "Task" to "Production Error".Aug 31 2023, 4:02 PM
Michael updated the task description. (Show Details)Aug 31 2023, 4:05 PM
Michael moved this task from Triaged Low (0-50) to Sorted Team A on the [DEPRECATED] wdwb-tech board.
Michael added a project: wmde-wikidata-tech.
ItamarWMDE moved this task from Incoming to WikibaseLexeme on the wmde-wikidata-tech board.Sep 12 2023, 1:00 PM
ItamarWMDE subscribed.Sep 12 2023, 1:43 PM

Task Prio

  • Affects end users / production (it's a production error)
  • Does not affect monitoring
  • Does not affect development
  • Does not affect onboarding
  • Affects additional stakeholders (WMF RelEng)
ItamarWMDE moved this task from WikibaseLexeme to [DOT] Prioritized on the wmde-wikidata-tech board.Sep 12 2023, 1:44 PM
ItamarWMDE added a project: Wikidata Dev Team.
ItamarWMDE moved this task from Incoming to [DOT] Tech Backlog on the Wikidata Dev Team board.Sep 12 2023, 1:53 PM
karapayneWMDE moved this task from [DOT] Tech Backlog to Unified DOT Backlog on the Wikidata Dev Team board.Sep 14 2023, 8:47 AM
ItamarWMDE renamed this task from Figure out why LexemeEditEntityTest::testGivenInvalidDataInClearRequest_errorIsReported is throwing an Exception/logging an error with AbuseFilter enabled to [LEX] Figure out why LexemeEditEntityTest::testGivenInvalidDataInClearRequest_errorIsReported is throwing an Exception/logging an error with AbuseFilter enabled.Sep 20 2023, 2:51 PM
ItamarWMDE renamed this task from [LEX] Figure out why LexemeEditEntityTest::testGivenInvalidDataInClearRequest_errorIsReported is throwing an Exception/logging an error with AbuseFilter enabled to [LEX] [TECH] Figure out why LexemeEditEntityTest::testGivenInvalidDataInClearRequest_errorIsReported is throwing an Exception/logging an error with AbuseFilter enabled.
ItamarWMDE moved this task from [DOT] Prioritized to Ongoing on the wmde-wikidata-tech board.Oct 11 2023, 10:53 AM
hoo claimed this task.Jan 23 2024, 6:56 PM
hoo moved this task from Unified DOT Backlog to Wikidata.org Slice on the Wikidata Dev Team board.
hoo edited projects, added Wikidata Dev Team (Wikidata.org Slice); removed Wikidata Dev Team.
hoo moved this task from In Task Breakdown to In Development on the Wikidata Dev Team (Wikidata.org Slice) board.
hoo added a comment.Jan 24 2024, 1:44 PM

Last seen in production on 2023-11-11 21:05:24.503994 (and twice before on that day, no further occurrences on mwlog1002).

All these three cases were API requests (by the same user) done via Special:ApiSandbox using the following parameters (queried from Hive's event.mediawiki_api_request table):

{"format":"jsonfm","data":"{\"forms\":[]}","formatversion":"2","wrappedhtml":"1","token":"[redacted]","new":"lexeme","bot":"1","action":"wbeditentity"}

With these exact API parameters I was able to reproduce the problem on Wikidata (hence there are two further production errors today).

gerritbot added a comment.Jan 24 2024, 4:22 PM

Change 992753 had a related patch set uploaded (by Hoo man; author: Hoo man):

[mediawiki/extensions/Wikibase@master] EntityContentDataCodec: Don't log exceptions

https://gerrit.wikimedia.org/r/992753

gerritbot added a project: Patch-For-Review.Jan 24 2024, 4:22 PM
hoo moved this task from In Development to Ready for Peer Review on the Wikidata Dev Team (Wikidata.org Slice) board.Jan 24 2024, 4:26 PM
gerritbot added a comment.Jan 24 2024, 5:01 PM

Change 992753 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] EntityContentDataCodec: Don't log exceptions

https://gerrit.wikimedia.org/r/992753

Maintenance_bot removed a project: Patch-For-Review.Jan 24 2024, 5:32 PM
gerritbot added a comment.Jan 24 2024, 6:11 PM

Change 992791 had a related patch set uploaded (by Hoo man; author: Hoo man):

[mediawiki/extensions/WikibaseLexeme@master] Revert "Disable exception logger for a test"

https://gerrit.wikimedia.org/r/992791

gerritbot added a project: Patch-For-Review.Jan 24 2024, 6:11 PM
ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.16; 2024-01-30).Jan 24 2024, 7:01 PM
gerritbot added a comment.Jan 25 2024, 9:23 AM

Change 992791 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexeme@master] Revert "Disable exception logger for a test"

https://gerrit.wikimedia.org/r/992791

Maintenance_bot removed a project: Patch-For-Review.Jan 25 2024, 9:31 AM
Lucas_Werkmeister_WMDE moved this task from Ready for Peer Review to Ready for Tech Verification on the Wikidata Dev Team (Wikidata.org Slice) board.Jan 25 2024, 12:36 PM
Lucas_Werkmeister_WMDE mentioned this in T326768: [LEX] OutOfRangeException: Lexeme L136328 doesn't have Form L136328-F2.Jan 26 2024, 5:00 PM
ArthurTaylor closed this task as Resolved.Feb 1 2024, 11:52 AM
ArthurTaylor moved this task from Ready for Tech Verification to Done on the Wikidata Dev Team (Wikidata.org Slice) board.
ArthurTaylor subscribed.

Submitted the API request again today (after the roll-out of 1.42.0-wmf.16) and have not seen the message again in Logstash. This is resolved.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits