User:Faebot

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Bot
This user account is a bot operated by Fæ (talk). It is not a sock puppet, but rather an automated or semi-automated account for making repetitive edits that would be extremely tedious to do manually.
Administrators: if this bot is malfunctioning or causing harm, please block it.

tasks • contribs • count • logs • page moves • block user • block log • flag log • global contribs • flag bot


Scope
Non-controversial categorization, interlinks, standard template-ing (such as mass licensing template changes or application of the artwork template) and corrections for large Wikimedia Commons projects (envisaged to be mostly related to Wikimedia UK initiatives, though these are often with international involvement) with supporting Commons project pages. For example:
GLAM related batch uploads and well specified and non-controversial mass corrections, categorization, licensing, interlinks or standard template improvements. For example:
Where there is any doubt as to whether a large systematic change would be controversial, these would be put up for a community consensus, in the first instance on the most relevant project page for interested community members to comment on and suggest improvements.
Extensions to original scope:
For the nature of activities, the bot will remain supervised, though it may be left for longer runs (500 images) and only on non-controversial changes after a significant monitored stable run.

Automatic or manually assisted:

Automatic (sample checks should only be needed as the images have been reviewed at source)

Edit type (e.g. Continuous, daily, one time run):

One time runs.

Maximum edit rate (eg edits per minute):

Around 6 edits per minute.
Bot flag request at Bots/Requests, raised in August 2011 and granted September 2012.

Log 2020

[edit]

Log 2019

[edit]
  • CDC videos a long term upload project moved to WMF labs, and newly under Faebot account, after the heavy video recoding needing was over demanding for an old laptop.
  • Wayback addition of Wayback Machine links for selected large GLAM related batch uploads.

Log 2018

[edit]

Log 2017

[edit]

Log 2014

[edit]

Wikimedia UK supported projects

[edit]

A live report of uploads can be found using this catscan2 report. Some reference tables of ongoing uploads per month are available at User:Faebot/reports/catscan.

June 2014

[edit]
  • Upgrade all jpeg images in Photochrom prints to same resolution as tiff (around 3x larger) by local download of the tiff and recompiling as jpg. This will apply to around 6,000 images and will be slow as it will rely on home broadband for both tif download and jpg upload.
  • Update Fellows collection with an additional few thousand images and create Fellows animations after some Village Pump discussion on maximum working GIF sizes (23MP total).
  • Finalize Photochrom prints collection, including sub-categorization by country. Announcement on Commons Village Pump. Parallel discussion on the German Wikipedia. An update to GWT changes functionality and some significant post-upload housekeeping was systematically run.
  • Ongoing discussions with the Wellcome Digital Library and Cancer Research UK. It is likely that Faebot will be able to support these this summer.
  • The next tranche of Avionics uploads will rely on paying of forum membership. As my own payment for Wikimedia UK's membership was rejected by the Chief Executive, this is on hold, as proposals should be from a member of the charity.

May 2014

[edit]
  • Photochrom prints collection: 20R - commenced, xml generation on WMUK kit, though the GWToolset runs the file upload.
  • Images from Fellows (auctioneers): 0R - announcement pending by Andy.
  • Re-visit of the Library of Congress British Cartoon Prints collection in order to general a set of files that were previously skipped, primarily as they were over 100MB. Catscan report
  • NYPL maps upload exceeds 10,000 files. Notice posted at the Village pump to attract volunteers to start using them. --Fæ (talk) 10:37, 9 May 2014 (UTC)

April 2014

[edit]
Central Tibet manuscript illustration, c.1500. LACMA collection.
Update to my original upload of 20,000 photographs from the museum in 2013. Based on a request from PKM, a user well known for her GLAM work.
Experimenting with the Python "flickrapi" module made it possible to upload an additional 3,083 photographs from Flickr that were All Rights Reserved, but these images are covered by a specific email release. Though this is running as a direct upload, similar code may be useful on other projects to generate a dataset for use with the GWToolset.
Christmas Menu, Royal Army Medical Corps 1918. From the National Library of Ireland
A few hundred photographs had previously been loaded from this Flickrstream of high quality early photographs from Ireland. This was an update to add the latest 220 archive images, now the NLI appears to have completed this Flickr Commons project.
The release of 20,000 high quality map images by the New York Public Library was of interest by volunteers on email lists and the Village Pump, along with requests for a bot to help with consistent uploads. This is more complex than it first appears, as direct links to the high resolution tiffs cannot be automatically "scraped" from the public website. I signed up for access to the NYPL API, and use this to deduce the url to download from. This is limited to 10,000 transactions. See Commons:Batch uploading/NYPL Maps. Many of the images are greater than 100MB, these are currently being skipped. Solutions of either creating a Python chunk uploader module (so up to 1GB might be uploaded) or generating an XML dump to use the GLAMwiki toolset to perform the upload are being considered (the latter would be much faster). Note, the average file size is large at ~50MB, due to bandwidth constraints and the fact that skipped files still result in 100MB being downloaded (as the API does not provide file sizes), the upload rate is of the order of just 5 per hour. The uploads have been popular, with Commons volunteers helping to crop and rotate the images.
The GWToolset is now being used to complete this upload (up to 50 times faster), including the files previously skipped due to being over 100MB in file size. A category for images over 50,000,000 pixel resolution in this upload has been created as Commons does not create thumbnails for images over this extremely large resolution. There are likely to be a few hundred at this pixel size from the 20,000.
This upload was paused when a few hundred images over 50 megapixels started to degrade WMF server performance in rendering tiff thumbnails. Operations are looking at ways of managing the operational load that thumbnail creation causes. The upload was cautiously restarted using a minimal number of processing threads.
After a request on Commons:Bots/Work_requests I have created a "live" SVG format checker to mark vector files with embedded raster graphics with {{BadSVG}}. This identifies around 10 to 40 files per day by sampling the Commons upload log every hour, checking the source code of all SVG files over 5KB in size. As this seems non-controversial, it is running on the WMUK supported kit, though at some future point it may be re-written and migrate to WMFlabs (note an SVGbot used to run but lapsed back in 2010). The templated files are not marked as Wikimedia UK supported, doing so might be seen as controversial as Wikimedia UK kit did not upload the files. Note, this is currently running inconsistently due to an unsolved memory leak problem.
  • A request from Robin for English Wikimedia articles to be automatically created for SSSIs in Wales has resulted in Wikipedia:Bots/Requests_for_approval#Faebot. This may not get approved, how we stage these potential 900 articles in a non-controversial way is under discussion.
  • Due to other priorities on time, the Geograph project has been delayed. This will take some experimentation and research to decide how to approach the next level of place identification and whether to mass update from the Geograph project.

March 2014

[edit]
  • Batch uploads of Collections in the Library of Congress started. Some of the tiff files are very high resolution scans of width greater than 5,000 pixels and filesize larger than 100MB. These include some collections of direct relevance to UK history. Most relevant to the UK are:
    • British Cartoon Prints Collection: 1017
      1780—1830 historic political cartoons
    • World War I posters in the Library of Congress: 1871
      As only a minority are British, these are not currently marked as sponsored by the chapter. Some posters, such as the more extreme propaganda posters from Germany, may have controversial content, however feedback from WikiProject MILHIST has been entirely supportive.

February 2014

[edit]
  • A backlog of more than 10,000 photographs as part of the Aircraft forum images were uploaded. This included around 5,000 from a Russian aircraft forum. These and later photographs will be identified as supported by Wikimedia UK. (Source code)
  • Images_from_MoD_uploaded_by_Fæ: 6032
    • Currently the Ministry of Defence releases around 35 to 70 high quality official photographs each month. Faebot should check the MoD site for new files daily, providing an estimated 600 high quality educational photographs each year. Fæ is in contact with the MoD Images library who will update him on changes in their API (which provides the metadata).

Initial

[edit]

In January 2014 the WMUK charity supplied a macmini to act as a server for selected Faebot projects this includes:

Log 2013

[edit]

Log 2012

[edit]

Awaiting batch upload completion

[edit]

Misc jobs

[edit]

Batch upload log

[edit]
_178 Category:Initials illuminated capitals added to sub-directories.
__76 Category:1873 Michigan County Maps high quality scans of detail county maps.
__24 Category:Black Panther Trial Sketches court sketches.
_100 Category:Randolph Linsly Simpson Collection portraits of black Americans from 1850s.
_144 Category:Cartes de visite Added Cartes de visite from the Beinecke Library.
_180 Category:New Mexico postcards by Tichnor Brothers Uploaded after confirming licence at Commons:Village pump/Copyright, 1930s-1945 postcards of New Mexico.
_300 Uploads from Ruslan Flickr 'Old photos and pictures' set, ~300 images mostly with pre-1917 Russian context {{PD-RusEmpire}} may apply
_100 Category:Ex libris ±100 decorative book plates and stamps (mainly 19th century)
_182 Category:Winslow Homer wood engravings prints 1858-1873, BPL
_125 Category:Across the Continent on the Kansas Pacific Railway (Route of the 35th Parallel), with photographs published in 1869, BPL
_700 Category:Animal Locomotion 11 Volumes by Eadweard Muybridge held by the BPL, 700 sets of photographs
1246 Category:Chromolithographs at Boston Public Library 1,200 high quality scans of 19th century popular images
Retrieved from "https://commons.wikimedia.org/w/index.php?title=User:Faebot&oldid=392340313"