Sounds like the patch will fix the problem with a gadget on English Wiktionary. MediaWiki:Gadget-aWa.js moves the sections of concluded discussions to talk pages of entries. It currently displays [subscribe] at the beginning of the section name, because it grabs the textContent of mw-headline elements.

I went through the reproduction steps and spotted the Edittools gadget erroring out, but I've fixed the checks for missing elements and that error has been cleared up. That gadget is installed by default for all logged-in users, so it could have been erroring out and causing some other JavaScript to not run. (I also had to fix one of my scripts, but that isn't relevant to most users.) Now the Edittools error doesn't appear in the JavaScript console, but it's still impossible to expand the heading, so the cause is something else.

In T354139#9492107, @Theknightwho wrote:

To me, the sortkey field for categories should be treated as nowiki text. As with nowiki, HTML entities would be decoded (i.e. & --> &) , but that's it.

I looked back at my first post and got the test.sh to run in toolforge-jobs when I set PATH=/data/project/rustup/rustup/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin:$PATH and removed stable from the command in test.sh. Maybe that means I can actually compile with toolforge-jobs now.

I updated the task description to link to the latest commit (with line number adjustments). Then I realized the post mentioned 1.40 so I changed the links to the 1.40-tagged commit.

Reopened because English Wiktionary is not English Wikiversity. If there is another reason that this is a duplicate, please explain.

In T49137#9165140, @Verdy_p wrote:

I have just discovered that table.sort(t) on a simple table of UTF-8 string, does NOT sort the table correctly (producing nearly random order).

This is apparently caused by a bug in the strcoll C function in the C.UTF-8 locale in the old version of glibc available on the Wikimedia servers, as described in T193096#4161287. Lua's table.sort uses the less-than (<) operator if no comparison function is provided (https://www.lua.org/source/5.1/ltablib.c.html#sort_comp), and the less-than operator uses strcoll when comparing two strings (https://www.lua.org/source/5.1/lvm.c.html#l_strcmp). In English Wiktionary, a module that regularly encounters non-BMP characters (> U FFFF) uses byte-by-byte comparison, which is equivalent to code point comparison for UTF-8.

In T50175#9164846, @Awesome_Aasim wrote:

There might actually be a better solution than this. If a module accesses category information, just prohibit population of categories by that module period. That is, if the module is like the first two scenarios, it would just do nothing and maybe add the module to a special page. This is something that can be checked after the module is done executing. For now, this might be a good solution until some way to detect these two scenarios is reliably determined. It would allow for category editnotices to work while preventing the two scenarios above from arising.

The explanation is that the display title {{DISPLAYTITLE:''M''_🜨}} was placed before the {{head|mul|symbol|head=''M''_🜨}} template, and the module (Module:headword ) invoked by the template provided a conflicting display title <span class="Latn">M🜨</span> in Lua (frame:callParserFunction("DISPLAYTITLE", display_title)). The module gets a return value that is a warning that renders like Warning: Display title "<i>M</i>_🜨" overrides earlier display title "<span class="Latn">M🜨</span>"., but doesn't print it. It probably should.

I posted a bug report on the Miri repository because I don't know if it would be an error in that tool or in the Rust compiler itself.

I took the offending function and test, got its dependencies, and reduced it to this shorter code that triggers the same error:

Oops, I was running rustc 1.63.0-nightly (a6b8c6954 2022-06-03). I got the error from the original post when I upgraded to rustc 1.68.0-nightly (9e75dddf6 2023-01-15). So it might be a compiler bug.

I looked at the code and couldn't detect any problems. I'm guessing it's referring to the allocation of the input string, but that shouldn't actually be dropped. I don't know how to reproduce the error message. I ran cargo nightly miri test --all-features -- ip::sanitize_ip_recognizes_subpages_of_ipv6_address (after having to disable sccache) in the mwtitle directory and the offending test succeeded.

I finally checked the webservice and it was gridengine. Switched to kubernetes with

At the moment I can't use toolforge-jobs to compile the Rust programs that I use in this tool. They are compiled now, but if I make any changes I will have to recompile. cargo wasn't working inside a toolforge-jobs script in the enwikt-translations tool, so I assume it won't work here either, though I haven't tried it. See the comment explaining what I tried with cargo in enwikt-translations.

I used jsub a few times in this tool to generate a sqlite database to try to speed up the tool. That didn't work. Not sure if the Grid Engine is used in any other way in it.

I've run the task with jsub for now so that the website is up-to-date with the latest dump. I'll try it on toolforge-jobs again if anyone has an idea of how to get cargo to work.

I'm so far unable to use the Rust toolchain in /data/project/rustup/rustup/.cargo/bin in the toolforge-jobs environment from inside my run.sh script.

@Ladsgroup @ArielGlenn Thank you for adding linktarget.sql! I'm able to generate my wanted templates list once again.

@JJMC89 Thank you. That looks like it'll work, and I'll try that when the pages-articles.xml comes out.

In T319724#8327344, @komla wrote:

In T319724#8327314, @Erutuon wrote:

I got an email about this months ago, but didn't understand which parts of my work on Toolforge needed to be changed. I do jsub jobs and a webservice on this tool. The jsub part clearly needs to use toolforge-jobs, but how do I determine whether the webservice needs to be changed somehow?

if you do webservice status it should tell you whether the webservice is running on gridengine or kubernetes.
If it is on gridengine, take a look at this use case continuity table to see if a kubernetes-equivalent command works for you.
https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#Use_case_continuity

My jsub command looks like this:

I got an email about this months ago, but didn't understand which parts of my work on Toolforge needed to be changed. I do jsub jobs and a webservice on this tool. The jsub part clearly needs to use toolforge-jobs, but how do I determine whether the webservice needs to be changed somehow?

I can't generate my wanted templates list anymore. The list generator (written with this Rust library) relies on page.sql and templatelinks.sql and now requires linktarget.sql in order to work. The dumps should have been part of the roadmap for this database change, but I guess not many people write database queries against the dump files as I do, so it's a bit of an afterthought.

It was bothering me that siteinfo2-namespacesv2.json in the 2022-02-01 dump had two 2s in it. Seems like siteinfo-namespacesv2.json would be a less confusing name. But I looked at the code for the Dump and SiteInfoDump and SiteInfoV2Dump classes and couldn't figure out where the v2 is coming from, and maybe it's too late to change it.

In T300469#7663398, @Bugreporter wrote:

You can download and compile clang by yourself and use the locally compiled version when building other softwares.

As far as I understand the examples listed in T270542 they all talk about pretty much exactly what this is about: semi-automatically injecting arbitrary code into a place where it can be executed. While attack-scenarios are a bit more unlikely in case of Lua modules, there is no guarantee either. Would we really make ScribuntoContent::canPreloadContent() return true?

In T270542#7608091, @thiemowmde wrote:

The author and reviewers of the commit apparently didn't look for legitimate use-cases […]

Being an active Wikipedia volunteer since 2004 I can assure you, I did.

In T270542#7607613, @Ammarpad wrote:

This is now intentionally and explicitly allowed only in wikitext content model for security reasons cf. r26087d4. Whether it worked before or never did there's no point in keeping this open.

@Ammarpad : I don't think this is a duplicate of T270542 because this is about preload= not putting text in the edit box, not about preloadparams[]= leaving $1 unchanged.

@thiemowmde: ASCII is defined by Unicode to include code points U 0000-U 007F, and so here non-ASCII is referring to code points U 0080 and above.

Aha, thank you. I will see if I can fix the gadget. I only noticed the similarity of the edit summaries while posting here, and so it hadn't occurred to me to search for "restore to version". I had searched for bdi before and didn't find any matches in gadgets.

Unfortunately the message in Tech News for this week uses the Wikipedia: namespace prefix, when it should use the Project: prefix and be wikilinked. If anyone tries to visit this page title in any wiki where the local name of the Project namespace is not Wikipedia, they will be redirected to a nonexistent Wikipedia page. For instance a person attempting to visit Wikipedia:AutoWikiBrowser/CheckPageJSON on English Wiktionary will be redirected to https://en.wikipedia.org/wiki/AutoWikiBrowser/CheckPageJSON rather than the correct https://en.wiktionary.org/wiki/Project:AutoWikiBrowser/CheckPageJSON, which redirects to https://en.wiktionary.org/wiki/Wiktionary:AutoWikiBrowser/CheckPageJSON. I'd edit the Tech News page now, but it's already gone out to all the wikis. Can it be corrected now?

Tried to remove the no head temp tag from the most recent 4 edits of sniggered on English Wiktionary and got The changes could not be applied: The tag "no head temp" is not allowed to be removed. The tag is added by abuse filter 68 (in Special:Tags, it's "Defined by the software", "Applied manually by users and bots") so it's the same case as the original posting here.

I've modified MediaWiki:Scribunto-module-line on English Wiktionary to link to #L-<line_number> in action=view of the erroring module. That looks horrible in the JavaScript-generated backtrace (which shows the raw wikitext), but at least people can jump to one of the source lines mentioned in the backtrace now. I may rethink this and revert my edit.

I got the same error trying to view the page. If I remember right, the gobbledygook at the beginning was useful somehow the last time I posted about an error message like this, so here it is:

In T240678#6507420, @Ammarpad wrote:

In T240678#6506685, @Erutuon wrote:

mw.title.new("#")

It does not return an object, there is no object to return

Note on my change to the title: mw.title.new("#") returns an object, but accessing any of the expensive fields causes an error to be thrown in mw.title.lua. When I type the following lines one-by-one into the console, the field accesses on the title object all throw the same error:

In T194953#6185511, @bd808 wrote:

Is there a movement towards Rust in any significant segment of the Wikimedia movement?

The problem has disappeared for me (the search and replace dialog shows up again), maybe at the same time that diffs became monospace again.

I am not able to bring up the search and replace dialog in CodeEditor in the English Wiktionary modules that I'm currently editing, either by pressing Ctrl F or by clicking the icon in the toolbar. This is very troublesome and annoying because I normally constantly use the search and replace dialog to jump to the right place in the code.

In T234729#6093505, @CCicalese_WMF wrote:

@Erutuon more information to reproduce the error would be helpful.

I don't know why the task was marked as Done, but the bug remains for me at least.

Also encountered on English Wiktionary at Module:kum-decl:

@Aklapper: I've corrected and updated the description and the title because the error is probably a general MediaWiki thing: it occurs in English Wiktionary and Wikipedia.

I've been getting sporadic errors like [XjoBKQpAAD8AAIE3bO4AAAAN] 2020-02-04 23:41:29: Fatal exception of type "TypeError" after being redirected from a Special:MyLanguage subpage linked from Template:Databases on MediaWiki wiki. For instance, I clicked a link to Special:MyLanguage/Manual:Langlinks_table from the template, was redirected to Manual:Langlinks_table, and saw the error there. Reloading usually gets the page to display.

Okay, so an additional request must be sent to actually construct the Category objects.

My script seems no faster with the patch, even with categories = True in the call to APISite.preloadpages. Commented on the part of api.py where separate requests are being sent for each page's categories.

@zhuyifei1999 Yes. Thank you so much!

Just to confirm, this request for fish to be installed to the grid exec nodes right?

@bd808: Thanks for your help! If only I'd asked sooner, this spring when the problem arose. :-)

It seems the reason is that the search engine doesn't search every page. Probably it times out at a certain point when searching through mainspace pages, and if no the Anatolian hieroglyphs are found by that point, no results are shown.

I renamed the task because for me, and presumably for others, the link doesn't jump to the top of the source code either. Perhaps something in my settings prevents the link from going to the top of the source code.

In T234729#5550855, @Aklapper wrote:

@Erutuon: Please provide a link that allows others to reproduce the problem. Thanks.