Page MenuHomePhabricator

Create instrumentation spec for Zebra #9 prototype
Closed, ResolvedPublic

Description

Background

We would like to test the Zebra #9 prototype and compare it to the existing Vector 2022 page layout

Acceptance criteria

Create an instrumentation specification for the following. Note which metrics we are already measuring and which will need to be added separately

  • pageviews
  • edit rates
  • session length
  • ToC usage
  • page tools menu usage

Filter by:

  • edit bucket
  • anon/logged-in(Only test on logged-in users)

Next steps in data process (bolding indicates step we are currently on):

  • PM and Data Analyst coordinate to identify experiment design, research questions, metrics and guardrails for the new feature and associated metrics.
  • Data analyst creates a list of events that need to be tracked to calculate each metric
  • Engineers review the event list and comment on whether we’re already tracking with existing instruments and if new instrumentation is required. New tickets created for any new instrumentation needed.
  • Data analyst works with engineers to figure out what the new instrumentation should look like and where to store the events.
  • Data analyst documents all of the above in the instrumentation spec
  • PM signs off on the plan

Instrumentation Spec

Documentation: Instrumentation Spec

Event Timeline

mpopov triaged this task as Medium priority.Mar 28 2023, 5:07 PM

Have listed events that need to be tracked to calculate each metric at the instrument spec for page layout AB test. Pass onto engineers to review.

jwang closed this task as Resolved.EditedMay 30 2023, 9:54 PM

Mark as resolved as the instrumentation spec has been finalized.
AB test launch will be tracked in T335972.
Adding skin key for mediawiki_web_ab_test_enrollment schema will be tracked in T335309.