Mentioned in SAL (#wikimedia-operations) [2024-10-24T17:04:52Z] <urbanecm> mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=nowiki (running as mw-script.codfw.ui7285yu; T376749)

I executed the script for nowiki. Logs:

This should be all done for nowiki!

Started checking the first 120 pages on the log. @EAkinloose will check the other 120 pages.
Progress can be found here: https://docs.google.com/spreadsheets/d/1PSusVBqaKtjiox9nsmQhj8y-cFLI9o2M2vx8JzzFkcQ/edit?gid=0#gid=0

There are some Flow pages that are moved to a sub-page that do not seem to have any Move notice and also the pages from which these Flow boards were moved do not have DiscussionTools enabled on them.

In T376749#10264513, @Ryasmeen wrote:

Progress can be found here: https://docs.google.com/spreadsheets/d/1PSusVBqaKtjiox9nsmQhj8y-cFLI9o2M2vx8JzzFkcQ/edit?gid=0#gid=0

Does this have to be an access restricted Google Doc, or could community members take a look?

In T376749#10264772, @Aklapper wrote:

In T376749#10264513, @Ryasmeen wrote:

Progress can be found here: https://docs.google.com/spreadsheets/d/1PSusVBqaKtjiox9nsmQhj8y-cFLI9o2M2vx8JzzFkcQ/edit?gid=0#gid=0

Does this have to be an access restricted Google Doc, or could community members take a look?

Thank you for raising this question, @Aklapper. For now, we're going to keep this doc private simply because we're working to meet a deadline and would like to defer the effort of reviewing the doc to ensure all the info. within it is fit for public view.

With the above said, if/when someone expresses a need to see the doc, we're happy to prioritize the clean-up necessary to make it visible.

Mentioned in SAL (#wikimedia-operations) [2024-10-31T23:41:15Z] <urbanecm> Run extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php for several wikis (T376749; wiki list is on task)

I'm running the script now per Editing request. Progress below:

• commonswiki (NO OUTPUT)
• cswiki (OK)
• enwiki (ERRORED OUT)
• eowiki (NO MOVES)
• frwiktionary
• hewiki
• kkwiki
• metawiki (ERRORED OUT)
• pawiki
• ruwiki (ERRORED OUT)
• specieswiki
• svwiki
• test2wiki
• tewiki
• outreachwiki

Highlights

I noticed some weird things while running the script. I note them below, full logs are linked near the bottom of my comment.

enwiki run errored out:

[urbanecm@deploy2002 ~/flow-logs]$ ./run.sh enwiki --dry-run
⏳ Starting extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php on Kubernetes as job mw-script.codfw.jnq84brc ...
⏳ Waiting for the container to start...
🚀 Job is running.
📜 Streaming logs:
The "Flow" extension must be installed for this script to run. Please enable it and then try again.
[urbanecm@deploy2002 ~/flow-logs]$

Flow is also not present at metawiki or ruwiki.

For commons, the script produced no output at all:

[urbanecm@deploy2002 ~/flow-logs]$ ./run.sh commonswiki --dry-run
⏳ Starting extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php on Kubernetes as job mw-script.codfw.k0wn6hmu ...
⏳ Waiting for the container to start...
🚀 Job is running.
📜 Streaming logs:


[urbanecm@deploy2002 ~/flow-logs]$

Although the dry run didn't print any results, it did take non-trivial time to finish:

[urbanecm@deploy2002 ~/flow-logs]$ time mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=commonswiki
⏳ Starting extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php on Kubernetes as job mw-script.codfw.eaq566fp ...
⏳ Waiting for the container to start...
🚀 Job is running.
📜 Streaming logs:

real    0m59.116s
user    0m5.823s
sys     0m0.656s
[urbanecm@deploy2002 ~/flow-logs]$

Is this OK?

Logs

I published complete logs of the script runs as Phabricator pastes, see links below:

commonswiki.log https://phabricator.wikimedia.org/P70811
cswiki.log https://phabricator.wikimedia.org/P70812
enwiki.log https://phabricator.wikimedia.org/P70813
eowiki.log https://phabricator.wikimedia.org/P70814
frwiktionary.log https://phabricator.wikimedia.org/P70815
hewiki.log https://phabricator.wikimedia.org/P70816
kkwiki.log https://phabricator.wikimedia.org/P70817
metawiki.log https://phabricator.wikimedia.org/P70818
outreachwiki.log https://phabricator.wikimedia.org/P70819
pawiki.log https://phabricator.wikimedia.org/P70820
ruwiki.log https://phabricator.wikimedia.org/P70821
specieswiki.log https://phabricator.wikimedia.org/P70822
svwiki.log https://phabricator.wikimedia.org/P70823
svwik.log https://phabricator.wikimedia.org/P70824
test2wiki.log https://phabricator.wikimedia.org/P70825
tewiki.log https://phabricator.wikimedia.org/P70826

run.sh

For transparency, here is run.sh:

[urbanecm@deploy2002 ~/flow-logs]$ cat run.sh 
#!/bin/bash

if [ "$#" -eq 0 ]; then
        echo "ERROR: First argument needs to be the wiki name"
        exit 1
fi

WIKI=$1
shift

echo "$ mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=$WIKI $@" > $WIKI.log
mwscript-k8s -f extensions/Flow/maintenance/FlowMoveBoardsToSubpages.php -- --wiki=$WIKI $@ 2>&1 | tee -a $WIKI.log
[urbanecm@deploy2002 ~/flow-logs]$

So, checking in to this...

These wikis were disabled by removing them from flow.dblist in our config repo on the mentioned dates. The effect of this is that the Flow extension isn't even loaded on those wikis, and so they've been completely unable to display any hypothetical existing Flow boards remaining since then. (But I'd assume that manual cleanup happened before we turned them off.)

commonswiki... the initial queries for number-of-flow-boards thinks that there should be 2 boards on that wiki, but they could potentially not be attached to existing pages which would produce that result. As for it taking so long... we're doing a query on page where page_content_model is CONTENT_MODEL_FLOW_BOARD. Since I don't think page_content_model has any index, I'd assume it's just that commons has a large page table to scan through. If enwiki hadn't errored out early because of Flow not being enabled, I'd assume it'd have taken a fair while as well -- though commons has twice as many pages, so it's probably the single worst-case wiki for that query.

In T376749#10283307, @DLynch wrote:

(But I'd assume that manual cleanup happened before we turned them off.)

Yes, confirmed. The Enwiki locations that it was deployed to, were each replaced with a simple wikitext-export.

For information about future phases, please see: Flow Deprecation Timeline

This information is not accessible for anyone not at the WMF (btw thats the majority of people in the wikimedia movement). Could you make it more accessible please?

In T376749#10286228, @Zabe wrote:

For information about future phases, please see: Flow Deprecation Timeline

This information is not accessible for anyone not at the WMF (btw thats the majority of people in the wikimedia movement). Could you make it more accessible please?

Thank you for the note, @Zabe. The information from that link is already public, but the description (somewhat unhelpfully) only links the private spreadsheet. You can find the deprecation timeline outlined at
https://www.mediawiki.org/wiki/Structured_Discussions/Deprecation#Deprecation_timeline.

In addition to the timeline, the spreadsheet also includes Flow usage data (which were used to decide which wikis belong to phase 0, which got migrated as part of this task). The usage data were compiled in T352174 and published under https://people.wikimedia.org/~urbanecm/growth-team/T352174_flow_per_day_per_page_20231206.html.

Hope this helps!

In T376749#10287086, @Urbanecm_WMF wrote:

In T376749#10286228, @Zabe wrote:

For information about future phases, please see: Flow Deprecation Timeline

This information is not accessible for anyone not at the WMF (btw thats the majority of people in the wikimedia movement). Could you make it more accessible please?

Thank you for the note, @Zabe. The information from that link is already public, but the description (somewhat unhelpfully) only links the private spreadsheet. You can find the deprecation timeline outlined at
https://www.mediawiki.org/wiki/Structured_Discussions/Deprecation#Deprecation_timeline.

In addition to the timeline, the spreadsheet also includes Flow usage data (which were used to decide which wikis belong to phase 0, which got migrated as part of this task). The usage data were compiled in T352174 and published under https://people.wikimedia.org/~urbanecm/growth-team/T352174_flow_per_day_per_page_20231206.html.

Hope this helps!

Yes thank!

Per today's team standup, @EAkinloose and @Ryasmeen will test 10% of the pages that were moved in the 31 October run and document findings in the existing Streaming logs sheet.

Completed testing the 10% of the pages that were moved in the 31st October run. The only issue I found during testing this task was: T378392 which is fixed now.