Page MenuHomePhabricator

Some history views and diffs unavailable on Wikipedias (Fatal ParameterAssertionException: Bad value for parameter $dbkey)
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error

Request URLs:

Request ID: XOTEygpAICEAALqO5IYAAADT

Opening of this link:
https://ru.wikipedia.org/w/index.php?title=Википедия:Запросы_к_администраторам&oldid=99570232

Results in error:
[XNBV-wpAMFMAAEjwO2cAAABC] 2019-05-06 15:42:55: Неустранимое исключение типа «Wikimedia\Assert\ParameterAssertionException»
This should not happen.

2019-05-06 15:42:55 [XNBV-wpAMFMAAEjwO2cAAABC] mw1248 ruwiki 1.34.0-wmf.3 exception ERROR: [XNBV-wpAMFMAAEjwO2cAAABC] /w/index.php?title=Википедия:Запросы_к_администраторам&oldid=99570232   Wikimedia\Assert\ParameterAssertionException from line 63 of /srv/mediawiki/php-1.34.0-wmf.3/vendor/wikimedia/assert/src/Assert.php: Bad value for parameter $dbkey: should not be empty unless namespace is main and fragment is non-empty {"exception_id":"XNBV-wpAMFMAAEjwO2cAAABC","exception_url":"/w/index.php?title=Википедия:Запросы_к_администраторам&oldid=99570232","caught_by":"mwe_handler"} 
[Exception Wikimedia\Assert\ParameterAssertionException] (/srv/mediawiki/php-1.34.0-wmf.3/vendor/wikimedia/assert/src/Assert.php:63) Bad value for parameter $dbkey: should not be empty unless namespace is main and fragment is non-empty
  #0 /srv/mediawiki/php-1.34.0-wmf.3/includes/title/TitleValue.php(107): Wikimedia\Assert\Assert::parameter(boolean, string, string)
  #1 /srv/mediawiki/php-1.34.0-wmf.3/includes/Linker.php(1206): TitleValue->__construct(integer, string, string)
  #2 [internal function]: Linker::{closure}(array)
  #3 /srv/mediawiki/php-1.34.0-wmf.3/includes/Linker.php(1233): preg_replace_callback(string, Closure, string)
  #4 /srv/mediawiki/php-1.34.0-wmf.3/includes/Linker.php(1135): Linker::formatAutocomments(string, Title, boolean, NULL)
  #5 /srv/mediawiki/php-1.34.0-wmf.3/includes/Linker.php(1501): Linker::formatComment(string, Title, boolean, NULL)
  #6 /srv/mediawiki/php-1.34.0-wmf.3/includes/Linker.php(1532): Linker::commentBlock(string, Title, boolean, NULL, boolean)
  #7 /srv/mediawiki/php-1.34.0-wmf.3/includes/page/Article.php(1567): Linker::revComment(Revision, boolean, boolean)
  #8 /srv/mediawiki/php-1.34.0-wmf.3/includes/page/Article.php(715): Article->setOldSubtitle(integer)
  #9 /srv/mediawiki/php-1.34.0-wmf.3/includes/actions/ViewAction.php(68): Article->view()
  #10 /srv/mediawiki/php-1.34.0-wmf.3/includes/MediaWiki.php(499): ViewAction->show()
  #11 /srv/mediawiki/php-1.34.0-wmf.3/includes/MediaWiki.php(294): MediaWiki->performAction(Article, Title)
  #12 /srv/mediawiki/php-1.34.0-wmf.3/includes/MediaWiki.php(865): MediaWiki->performRequest()
  #13 /srv/mediawiki/php-1.34.0-wmf.3/includes/MediaWiki.php(515): MediaWiki->main()
  #14 /srv/mediawiki/php-1.34.0-wmf.3/index.php(42): MediaWiki->run()
  #15 /srv/mediawiki/w/index.php(3): require(string)
  #16 {main}

From T222857:

Examples:

https://zh.wikipedia.org/w/index.php?title=萨塞克斯公爵夫人梅根&diff=54343065&oldid=54339003

Error: [XNO8wQpAIC0AAFgBaJMAAAAX] 2019-05-09 05:38:09: 类型“Wikimedia\Assert\ParameterTypeException”的致命例外

https://zh.wikipedia.org/w/index.php?title=秋元康&curid=1069252&diff=54302740&oldid=54064765

Error: [XNO8IwpAICEAAE@gG9QAAAAY] 2019-05-09 05:35:31: 类型“Wikimedia\Assert\ParameterTypeException”的致命例外

Impact

Internal error is shown!

Notes

Affects various pages

https://logstash.wikimedia.org/goto/8c9d3b83322b23305122be6603222246

First occurence is from April 30th with number of occurrences varying:

nsexception.png (235×1 px, 11 KB)

Caused by:

34582db81c9418a12721df58081c02bb05d2660a / https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /504000/ ???

Related Objects

Mentioned In
T225585: Unable to open user sub pages with trailing space (Blank page fatal: "invalid DB key")
T224811: Special:Log/block inaccessible on several wikis with fatal ParameterAssertionException "Bad value for parameter $dbkey: should not be empty"
T224664: Internal error when trying to see the difference between revisions [Wikimedia\Assert\ParameterAssertionException]
T224095: Bad value for parameter $dbkey: should not be empty unless namespace is main and fragment is non-empty
T222857: Some history views and diffs unavailable on zh.wikipedia.org (Fatal ParameterTypeException: Bad value for parameter $fragment)
T223197: Wikimedia Assert ParameterTypeException when rendering permanent link pages
Mentioned Here
T224664: Internal error when trying to see the difference between revisions [Wikimedia\Assert\ParameterAssertionException]
rMW66e78a53c35e: Linker: Fix empty auto-summaries triggering a fatal error
T224095: Bad value for parameter $dbkey: should not be empty unless namespace is main and fragment is non-empty
rMWc735a545df3a: Linker: Fix empty auto-summaries triggering a fatal error
rMW34582db81c94: Allow fragment-only TitleValues
rMWb6e1e99bec8d: Use LinkTarget in Linker instead of Title
T222857: Some history views and diffs unavailable on zh.wikipedia.org (Fatal ParameterTypeException: Bad value for parameter $fragment)
T200055: Some pages have revisions by invalid usernames (with leading or trailing space) which break the "Info" and "History" views
T222529: Wikimedia\Assert\ParameterAssertionException when rendering a log snippet and log_user_text is empty

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I am seeing what might be the same error on the English Wiktionary. The diff for what is currently the latest revision of MediaWiki:Common.css shows the error [XNPAewpAAEMAADungXAAAAAS] 2019-05-09 05:54:03: Fatal exception of type "Wikimedia\Assert\ParameterTypeException". I'm the author of both revisions and my username is not invalid, so this seems not to be an example of T200055.

I think this ticket is the same as T222857, because the revisions has an empty autocomment.

Change 509396 had a related patch set uploaded (by 星耀晨曦; owner: 星耀晨曦):
[mediawiki/core@master] Checks the autocomment whether is empty in Linker::formatAutocomments()

https://gerrit.wikimedia.org/r/509396

https://de.wikipedia.org/wiki/Spezial:Diff/74865827/107930065?title=Benutzer:Helge_Sternke

[XNYHbgpAIDAAAJ4sk7oAAACC] 2019-05-10 23:21:18: Fataler Ausnahmefehler des Typs „Wikimedia\Assert\ParameterTypeException“

? ? ?

@Bestoernesto: Do you have a question, or how to interpret your comment?

I don't have any knowledge of data processing, but I believe that my problem may be due to a similar software bug as the examples above. When I try to look at this version comparison (link above), I get the error message copied below. However, this is not a general error, but only affects the user page of this one user.

https://zh.wikipedia.org/w/index.php?title=阮福映&diff=54331203&oldid=53920231
[XNjH2gpAICsAADKpdOEAAAAT] 2019-05-13 01:26:50: 类型“Wikimedia\Assert\ParameterAssertionException”的致命例外

https://zh.wikipedia.org/w/index.php?title=阮福映&oldid=53920231
[XNjIyApAAEQAAGj4BOwAAADI] 2019-05-13 01:30:48: 类型“Wikimedia\Assert\ParameterAssertionException”的致命例外

https://zh.wikipedia.org/w/index.php?title=提帮功王子&oldid=54305820
[XNjI1ApAAD0AAAqLeDsAAAAB] 2019-05-13 01:31:00: 类型“Wikimedia\Assert\ParameterAssertionException”的致命例外

and more

on zhwiki, it seems that all the edit summary that contain "/\*\s*\*/" have this problem.

Krinkle renamed this task from Internal error: "Wikimedia\Assert\ParameterAssertionException" in ruwiki: "Bad value for parameter $dbkey: should not be empty" to Some history views and diffs unavailable on ru.wikipedia.org (Fatal ParameterAssertionException: Bad value for parameter $dbkey).May 20 2019, 5:58 PM
Krinkle raised the priority of this task from High to Unbreak Now!.May 20 2019, 6:07 PM
Krinkle added a project: Platform Engineering.
Krinkle added subscribers: Simetrical, daniel, Krinkle.

Most likely caused by b6e1e99bec8 (April 2019, two months ago), which switched the method from Title::makeTitleSafe to TitleValue. The TitleValue here does not behave the same. It throws fatal for any non-string $fragment (whereas here it can sometimes be false), and also throws fatal for any case of where both dbkey and $fragment are empty (in NS_MAIN).

Those checks are fair when the use case is to create a wiki link with pre-validated input. But that is not the case here. The edit summary logic explicitly is arbitrary user input. It is meant to render as plain text, unless it happens to be a valid link. This was lost in b6e1e99bec8. We now have dead code for acting on the case where there is no title present, and on the line before that, we unexpectedly fatal for all such cases where there is no title given.

The user-facing impact is that users are unable to view the history of certain pages, and unable to open the diff of affected revisions.

The server-side impact is that these easily-discoverable urls constantly result in an uncacheable HTTP 500 Internal Server Error response, which is a risk in terms of traffic, and also can cause operational alarms or auto-abort unrelated deployments, which is actively affecting other teams.

Tagging, author, reviewer and related team (Core-Platform-Team) to follow-up ASAP so as to unbreak this.

Krinkle renamed this task from Some history views and diffs unavailable on ru.wikipedia.org (Fatal ParameterAssertionException: Bad value for parameter $dbkey) to Some history views and diffs unavailable on Wikipedias (Fatal ParameterAssertionException: Bad value for parameter $dbkey).May 20 2019, 6:45 PM
Krinkle updated the task description. (Show Details)

Change 511673 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/core@master] Fix empty auto-summaries triggering a fatal error.

https://gerrit.wikimedia.org/r/511673

There are various code path from extensions or mediawiki/core that ends up hitting the error. It has been added in mediawiki/core by https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /504000/

The first occurrences were on April 30th.

There are various code path from extensions or mediawiki/core that ends up hitting the error. It has been added in mediawiki/core by https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /504000/

The first occurrences were on April 30th.

Does this mean that there are the other callers using TitleValue class wrong? Before I only thought that there was a problem with calling Linker::formatAutocomments() of TitleValue class.

There are various code path from extensions or mediawiki/core that ends up hitting the error. It has been added in mediawiki/core by https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /504000/

That change actually relaxed the check, it didn't make it more restrictive. But I think it was made in the context of more code using TitleValue for more things, which in turn is causing the problem now.

Change 512502 had a related patch set uploaded (by Krinkle; owner: Daniel Kinzler):
[mediawiki/core@wmf/1.34.0-wmf.6] Linker: Fix empty auto-summaries triggering a fatal error

https://gerrit.wikimedia.org/r/512502

Change 512502 merged by jenkins-bot:
[mediawiki/core@wmf/1.34.0-wmf.6] Linker: Fix empty auto-summaries triggering a fatal error

https://gerrit.wikimedia.org/r/512502

Mentioned in SAL (#wikimedia-operations) [2019-05-25T22:00:05Z] <krinkle@deploy1001> Synchronized php-1.34.0-wmf.6/includes/Linker.php: T222628 / rMWc735a545df3a (duration: 00m 51s)

Krinkle removed a project: Patch-For-Review.

Thanks all. Confirmed in prod with all the given urls that were mentioned in this task, and T222857, and T224095.

Change 509396 abandoned by 星耀晨曦:
Checks the autocomment whether is empty in Linker::formatAutocomments()

Reason:
The bug fixed in rMW66e78a53c35e.

https://gerrit.wikimedia.org/r/509396

Change 512126 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/core@master] linker: Streamline Linker::formatAutocomments() and add tests

https://gerrit.wikimedia.org/r/512126

Reedy added a subscriber: DonRumata.
Reedy subscribed.

Because 512126 has not been merged yet and the fix for 1.34.0-wmf.6 branch was not ported to master branch, resulting in 1.34.0-wmf.7 branch not including the fix, resulting in bug regressions.

This will affect all wikis using 1.34.0-wmf.7, for now affect group0 and group1. Will soon affect group2. (thursday train at 12:00–13:00 UTC)

Change 513275 had a related patch set uploaded (by Reedy; owner: Daniel Kinzler):
[mediawiki/core@wmf/1.34.0-wmf.7] Linker: Fix empty auto-summaries triggering a fatal error

https://gerrit.wikimedia.org/r/513275

Reopened as it wasn't fixed in master (only in .6), meaning it's regressed in .7

I've cherry picked Daniels .6 patch into .7 and will deploy, as it worked before

Ideally, someone merges Thiemo's patch into master before .8 is cut next week

Change 513275 merged by jenkins-bot:
[mediawiki/core@wmf/1.34.0-wmf.7] Linker: Fix empty auto-summaries triggering a fatal error

https://gerrit.wikimedia.org/r/513275

Mentioned in SAL (#wikimedia-operations) [2019-05-30T12:58:46Z] <reedy@deploy1001> Synchronized php-1.34.0-wmf.7/includes/Linker.php: T222628 (duration: 01m 04s)

Mentioned in SAL (#wikimedia-operations) [2019-05-30T13:00:16Z] <reedy@deploy1001> Synchronized php-1.34.0-wmf.7/tests/phpunit/includes/: T222628 (duration: 01m 06s)

RazeSoldier lowered the priority of this task from Unbreak Now! to High.May 30 2019, 1:01 PM

Merge the fix on master branch is not urgent. Drop priority.

Downgrading to high but leaving open until patch is merged into master

Seems https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /511673/ was C 2 V 2 but not actually merged for some reason...

Change 512126 abandoned by Daniel Kinzler:
linker: Streamline Linker::formatAutocomments() and add tests

Reason:
Squashed into I05408ede0e20dfd976f4057fc5baab461d2ef769. Thanks for making the code more readable!

I suggested earlier to do it the other way around and squash my tests into your commit, but I guess you didn't see the comment. I hope it's ok for you this way as well. I'm attributing the Linker code to you in the commit message.

https://gerrit.wikimedia.org/r/512126

Seems https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /511673/ was C 2 V 2 but not actually merged for some reason...

Krinkle C 2'ed but then immediately edited the commit message without re-C 2'ing. That means the C 2 is retained by gerrit, but jenkins knows it applies to a different commit hash and so ignores it.

Change 511673 merged by jenkins-bot:
[mediawiki/core@master] Fix empty auto-summaries triggering a fatal error.

https://gerrit.wikimedia.org/r/511673

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM