Wikidata:Requests for permissions/Bot/William Avery Bot 9

From Wikidata
Jump to navigation Jump to search
The following discussion is closed. Please do not modify it. Subsequent comments should be made in a new section. A summary of the conclusions reached follows.
 Approved--Ymblanter (talk) 18:57, 9 August 2022 (UTC)[reply]

William Avery Bot 9

[edit]

William Avery Bot (talk • contribs • new items • new lexemes • SUL • Block log • User rights log • User rights • xtools)
Operator: William Avery (talk • contribs • logs)

Task/s: Remove tracking parameters from reference URLs, as suggested at Wikidata:Bot requests § Tracking parameters in reference URLs. I would like to run this as a recurring task, after clearing the c. 2800 current instances.

Code: cleanseRefs.py - pywikibot script. Task logic is in the treat_page method.

Function details:

Candidate items can be harvested using a database query on the externallinks database table.

I am using a list of tracking parameters from here, but the only ones I have so far detected on Wikidata items are fbclid, igshid, gclid and Urchin Tracking Module ("utm_") parameters

The URLs processed are the ones on the reference URL (P854) property of references.

Each URL is parsed using the standard python library urllib, to extract the parameters into a list. Any that are on the list of tracking parameters are removed. The URL is then reassembled using urllib.

After all the references on the item have been processed, the item is updated with the new URL values.

Testing: I have run the script on a few individual items, under my own account

I intend to run a bulk test on 4 August, if there are no objections. William Avery (talk) 10:53, 3 August 2022 (UTC)[reply]

Diffs from the test run are here

Observations:

  • During the test run I turned off 'strict' parsing of the URL parameters, because it was causing the module to throw an error.
  • The processing didn't break any links. A few were already broken.
  • URLs in archive URL (P1065) are untouched, and can contain tracking parameters. The archiving site are unlikely to make use of these parameters. See here for an example.
- William Avery (talk) 11:38, 4 August 2022 (UTC)[reply]

Discussion:

I have placed a link to this request at Property talk:P854 § Removal of tracking parameters from reference URLs. William Avery (talk) 11:38, 4 August 2022 (UTC)[reply]

The above discussion is preserved as an archive. Please do not modify it. Subsequent comments should be made in a new section.
Retrieved from "https://www.wikidata.org/w/index.php?title=Wikidata:Requests_for_permissions/Bot/William_Avery_Bot_9&oldid=1700277699"