@Mooeypoo has expressed concerns over tracking algorithmic bias around the Edit-Review-Improvements project. I think that @jmatazzoni and @Pginer-WMF should consider discussing this angle of algorithmic work in relation to ERI since it's a serious user-facing concern.

When I presented on my preliminary results regarding anonymous editor bias in damage detection, @Tbayer raised some good counter-points. I wonder if he'd want to bring that perspective to the dev summit.

I've been working on this idea around T148700: JADE: UI/API for reviewing/refuting how ORES classifies you and your stuff

From a blog I wrote about the idea:

So, I was listening to an NPR show titled "Digging into Facebook's File on You". At some point, there was some casual discussion of laws that some countries in the European Union have re. users' ability to review and correct mistakes in data that is stored about them. This made me realize that ORES needs a good mechanism for you to review how the system classifies you and your stuff.

Sounds like a fascinating topic.

This is certainly of concern to me in search. I'm starting to evaluate ways to build ML re-ranking systems using user click through data as the input, and the ability of users to 'poison the well' so to speak will certainly become an issue to deal with. Would love to talk about how others are dealing with the issue of training with data from user behaviour.

In T147929#2807379, @Halfak wrote:

...

When I presented on my preliminary results regarding anonymous editor bias in damage detection, @Tbayer raised some good counter-points. I wonder if he'd want to bring that perspective to the dev summit.

I will be at the summit and am happy to participate in a session regarding this topic. (It's not my core work area currently, but I have been interested in it both as a volunteer editor who does quite of patrolling on Wikipedia and Wikidata, now with the help of ORES, and as a Wikipedia research topic in general - I also wrote a review of a related paper recently.)
In any case, I agree it's a worthwhile topic for a session at the summit.

This has been a hot topic in the media recently (e.g. Cathy O'Neil's book Weapons of Math Destruction has gotten at lot of attention from the press). I don't know a lot about it, but I would really enjoy the chance to learn more.

Complements T149373 quite nicely, IMO.

https://medium.com/@robot_MD/when-bias-in-product-design-means-life-or-death-ea3d16e3ddb2#.5a1ker5hd seems related.

To the owner of this session: Here is the link to the session guidelines page: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines. We encourage you to recruit Note-taker(s) 2(min) and 3(max), Remote Moderator, and Advocate (optional) on the spot before the beginning of your session. Instructions about each role player's task are outlined in the guidelines. The physical version of the role cards will be made available in all the session rooms. Good luck prepping, see you at the summit! :)

Just set up https://etherpad.wikimedia.org/p/devsummit17-AI_ethics. Looking forward to seeing you all in a few days.

I've added the link for the Youtube stream. See you all in person or on IRC tomorrow morning.

Note-taker(s) of this session: Follow the instructions here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines#NOTE-TAKER.28S.29 After the session, DO NOT FORGET to copy the relevant notes and summary into a new wiki page following the template here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Your_Session and also link this from the All Session Notes page: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/All_Session_Notes. The EtherPad links are also now linked from the Schedule page (https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Schedule) for you!

https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/AI_ethics