Track improvements made to mobile block messages
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Niharika
	Oct 1 2020, 6:19 PM

Description

Goal

We have made significant improvements to mobile block messages. We would like to track how much the new CTA is being shown and clicked and see if there are any improvements over the baseline (T261253).

Linked tasks:

Parent task for improving block messages: T189717: Better handle block reasons on mobile (specifically templates and HTML comments)
Logging added to the new block messages: T260218: Add logging for measuring impact of our work on improving the mobile block messages

Report

Dashboard : https://grafana.wikimedia.org/d/9k1-FuDGz/block-message-on-mobile?orgId=1&from=1600551069275&to=1603143069275

Related Objects

Mentioned In: T266880: Investigate cause of increase in mobile block messages shown
Mentioned Here: T189717: Better handle block reasons on mobile (specifically templates and HTML comments)
T260218: Add logging for measuring impact of our work on improving the mobile block messages
T261253: Baseline studies for block messages on mobile

Event Timeline

Niharika triaged this task as Medium priority.Oct 1 2020, 6:19 PM

Niharika created this task.

jwang claimed this task.Oct 1 2020, 10:48 PM

jwang moved this task from Triage to Current Quarter on the Product-Analytics board.

jwang edited projects, added Product-Analytics (Kanban); removed Product-Analytics.Oct 19 2020, 4:18 PM

I have built a dashboard on Grafana to monitor the impact of the improvement.

Summary ( by 2020-10-19)

With the improvement, the average daily block notices showed up is 17.8k, 4.9 times more than before.

Old feature

New feature

CTA click though rate is 4%

Top five wikis with most block notices are : enwiki, jawiki, ptwiki, ruwiki, thwiki, similar to the baseline.

Users access the special page or the expanded drawer to see the reason 2018 times by October 19, 2020. It seems the event did not happen every day. Need to check with R&D to see if it should be a daily event.