Page MenuHomePhabricator

Adjust edit count bucketing for TemplateData
Closed, ResolvedPublic

Description

We need to update the following for TemplateData:

  • Metric collection
  • Schema
  • Aggregation queries

This is a subtask of T269986

Event Timeline

Change 657566 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/extensions/TemplateData@master] Bucket the user's edit count before logging an event

https://gerrit.wikimedia.org/r/657566

Open questions:

  1. Do we gain any value by including edit count buckets in metrics collected using this https://meta.wikimedia.org/wiki/Schema:TemplateDataApi?
  2. How should we go about the now redundant user_edit_count field in https://meta.wikimedia.org/wiki/Schema:TemplateDataEditor?

Change 659227 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[analytics/reportupdater-queries@master] [WIP] Use the edit count bucket sent by TemplateData

https://gerrit.wikimedia.org/r/659227

awight moved this task from Doing to Review on the WMDE-TechWish (Sprint-2021-01-20) board.

Change 657566 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Bucket the user's edit count before logging an event

https://gerrit.wikimedia.org/r/657566

awight set the point value for this task to 3.Feb 2 2021, 12:05 PM
awight changed the point value for this task from 3 to 5.
Andrew-WMDE changed the point value for this task from 3 to 2.

Change 659227 merged by Mforns:
[analytics/reportupdater-queries@master] Use the edit count bucket sent by TemplateData

https://gerrit.wikimedia.org/r/659227

Change 666685 had a related patch set uploaded (by Awight; owner: Awight):
[analytics/reportupdater-queries@master] Fix bad table name in query

https://gerrit.wikimedia.org/r/666685

Change 666685 merged by Mforns:
[analytics/reportupdater-queries@master] Fix bad table name in query

https://gerrit.wikimedia.org/r/666685

Error logs:

Feb 25 10:00:00 an-launcher1002 systemd[1]: Started Report Updater job for templatedata.
Feb 25 10:00:00 an-launcher1002 kerberos-run-command[11705]: User analytics executes as user analytics the command ['/usr/bin/python3', '/srv/reportupdater/reportupdater/update_reports.py', '-l', 'info', '/srv/r
Feb 25 10:00:00 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:00,851 - INFO - Starting execution.
Feb 25 10:00:00 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:00,908 - INFO - Executing "<Report key=dialog type=script granularity=days lag=18000 is_funnel=True first_date=2020-12-14 start
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:15,701 - ERROR - Report "dialog" could not be executed because of error: object of type 'NoneType' has no len()
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: Traceback (most recent call last):
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 171, in normalize_results
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:     empty_row = [report.start]   [None] * (len(normalized_header) - 1)
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: TypeError: object of type 'NoneType' has no len()
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:15,729 - INFO - Executing "<Report key=actions type=script granularity=days lag=18000 is_funnel=True first_date=2020-12-14 star
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:27,827 - ERROR - Report "actions" could not be executed because of error: object of type 'NoneType' has no len()
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: Traceback (most recent call last):
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 171, in normalize_results
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:     empty_row = [report.start]   [None] * (len(normalized_header) - 1)
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: TypeError: object of type 'NoneType' has no len()
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:27,829 - INFO - Execution complete.
Feb 25 10:00:27 an-launcher1002 systemd[1]: reportupdater-templatedata.service: Succeeded.

Change 667565 had a related patch set uploaded (by Awight; owner: Awight):
[analytics/reportupdater-queries@master] Fix typo: no "performer" field

https://gerrit.wikimedia.org/r/667565

Change 667565 merged by Mforns:
[analytics/reportupdater-queries@master] Fix typo: no "performer" field

https://gerrit.wikimedia.org/r/667565

awight claimed this task.