Page MenuHomePhabricator

Identify wikis for mobile DiscussionTools A/B test
Closed, ResolvedPublic

Description

This task is about identifying the Wikipedias that will participate in the A/B test of the complete set of mobile DiscussionTools features: Reply Tool, New Topic Tool, Topic Containers, Clearer Affordances, and the new Page Frame.

There are two parts to this task:

  1. Identifying potential wikis for inclusion in the mobile DiscussionTools A/B test
  2. Confirming with these wikis that they are comfortable participating in the mobile DiscussionTools A/B test

Selection Criteria

The "Candidate wikis" ought to, collectively, meet the following requirements:

  • People, across experience levels, depend on mobile talk pages to communicate with peers
    • In aggregate, ≥ 2,000 unique people using mobile talk pages at the wikis included in the A/B test
    • ≥ 25 unique Junior Contributors using mobile talk pages at each of the wikis included in the A/B test
  • The test includes people from a range of cultures and geographies who communicate in a variety of languages and scripts
    • ≥ 15 distinct wikis
    • A minimum of one wiki whose volunteers communicate in an LTR language and one wiki whose volunteers communicate in an RTL language
    • A minimum of one wiki whose volunteers communicate in a non-Latin script
    • A minimum of one wikis where ≥ 10% of the unique people editing mobile talk pages each month are located in a country within East Asia
    • A minimum of one wikis where ≥ 10% of the unique people editing mobile talk pages each month are located in a country within Sub-Saharan Africa
  • Do NOT already have access to any of the mobile DiscussionTools (=Arabic, Bangla, Hebrew, Japanese, Korean, Vietnamese, French, Haitian Creole, and Chinese Wikipedias)

Candidate wikis

WikiLink to on-wiki conversationStatus (e.g. no objections)
eswikino objections
ptwikino objections
ruwikino objections
fawikino objections
dewikisupport
itwikino objections
trwikino objections
idwikisupport
plwikino objections
hiwikino objections
nlwikino objections
ukwikino objections
thwikino objections
azwikino objections
rowikino objections

Done

  • @Whatamidoing-WMF proposes and documents the wikis that meet the === Selection Criteria in the ===Candidate wikis section above
  • @MNeisler verifies the ===Candidate wikis will produce a sufficient amount and diversity of usage data for us to draw conclusions from
  • @Whatamidoing-WMF consults with the ===Candidate wikis and documents the wikis that are and are not open to participating in the A/B test
  • @ppelberg updates CONFIG CHANGE TICKET with the list of wikis confirmed to participate in the A/B test

Event Timeline

ppelberg moved this task from Backlog to Triaged on the DiscussionTools board.
ppelberg moved this task from Backlog to Community Engagement on the Editing-team (Tracking) board.
ppelberg updated the task description. (Show Details)

@MNeisler, I don't think we have a current count on how many wikis have 25 newcomers who used the mobile site to edit a talk page, but on the theory that wikis with more mobile-majority editors will also have more mobile newcomers, and the more newcomers, the more who will edit the talk page, then I suggest these 12 Wikipedias for this test:

eswiki
ptwiki
ruwiki
fawiki
dewiki
itwiki
trwiki
idwiki
plwiki
hiwiki
nlwiki
ukwiki

If this is much more than you need, then we could cut the list. The most important wikis in this list are fawiki and hiwiki (RTL and IME issues).

If more are needed, then I'd suggest adding these smaller or more desktop-focused wikis:

azwiki
huwiki
cswiki
thwiki
simplewiki
rowiki
elwiki
nowiki
mswiki

If many more editors are needed, then we should consider looking outside of the Wikipedias (e.g., Commons, Wikidata, English Wiktionary, Meta-Wiki, and MediaWiki.org).

Thanks @Whatamidoing-WMF! I took a look at the number of distinct mobile talk page editors at the 12 proposed wikis and confirmed that they are sufficient to produce a sufficient amount and diversity of usage data for the analysis (See stats summary and recommendation below).

AB Test Wiki Recommendation: (cc @ppelberg )
The 12 Wikipedias proposed in T314950#8190366 look good to me. I'd also recommend adding a few of the smaller wikis mentioned to increase the diversity of data and have at least 15. Specifically, I think thwiki, azwiki, and rowiki would be good as they seem to have a good proportion of distinct mobile Junior Contributors.

Stats for 12 proposed wikis (Based on edits logged for August 2022 in mediawiki_history)

  • Total number of distinct registered editors: 5,263 (meets the 2,000 distinct editor requirement). Note: We are not able to identify distinct anon editors so this just reflects registered editors.
  • Number of distinct editors by wiki: All wikis have at least 100 distinct registered editors, which is sufficient for per wiki analysis.
  • Number of distinct registered Junior Contributors: 2,634
  • Number of Junior Contributors per wiki: All wikis have 40 or more Junior Contributors that have made an edit to mobile (meets the ≥ 25 unique Junior Contributor requirement)
  • Wiki where ≥ 10% of the unique people editing mobile talk pages each month are located in a country within East Asia: Idwiki (rough estimate based on number of idwiki in the East, Southeast Asia and Pacific Region)
  • Wiki where ≥ 10% of the unique people editing mobile talk pages each month are located in a country within Sub-Saharan Africa: ptwiki (About 7% (28 unique mobile ptwiki editors) from Sub-Sahran Africa. A little below the10% requirement but I don't think we'll get that for Sub-Saharan Africa unless looking at enwiki)

I have left messages for all 15 Wikipedias (including thwiki, azwiki, and rowiki).

ppelberg updated the task description. (Show Details)