Page MenuHomePhabricator

FY2024-25 WE4.2.3 CAPTCHA evaluation frameworkGoal
ActivePublic

Members (3)

Watchers (1)

Details

Description

Tasks related to the WMF FY2024-25 WE4.2.3 hypothesis on CAPTCHAs:

If we build an evaluation framework using publicly available technologies similar to the ones used in previous attacks we will learn more about the efficacy of our current CAPTCHA at blocking attacks and could recommend a CAPTCHA replacement that brings a measurable improvement in terms of the attack rate achievable for a given time and financial cost.

Recent Activity

Tue, Jan 7

Aklapper closed T289607: <Security Initiative> Improving Captcha as Invalid.
Tue, Jan 7, 6:13 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Foundational Technology Requests

Mon, Jan 6

VirginiaPoundstone added a project to T379179: Send captcha API response data to event logging: Data-Engineering-Radar.
Mon, Jan 6, 9:10 PM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering
VirginiaPoundstone removed a project from T379179: Send captcha API response data to event logging: Experimentation Lab.
Mon, Jan 6, 8:50 PM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering
VirginiaPoundstone moved T379179: Send captcha API response data to event logging from TBD to Move to Data Engineering on the Experimentation Lab board.
Mon, Jan 6, 8:26 PM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering
kostajh added a comment to T379179: Send captcha API response data to event logging.

@acooper, what would be the producer of the captcha events? Is it code we write and own? If so, this should be easy. If not, we have ways, but it kind of depends on the structure of the events sent.

Mon, Jan 6, 1:17 PM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering

Sun, Jan 5

RhinosF1 added a member for FY2024-25 WE4.2.3 CAPTCHA evaluation framework: RhinosF1.
Sun, Jan 5, 6:24 PM

Mon, Dec 16

Ottomata added a comment to T379179: Send captcha API response data to event logging.

@acooper, what would be the producer of the captcha? Is it code we write and own? If so, this should be easy. If not, we have ways, but it kind of depends on the structure of the events sent.

Mon, Dec 16, 2:10 PM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering
acooper added a comment to T379179: Send captcha API response data to event logging.

We need to do something like this schema: https://schema.wikimedia.org/repositories/secondary/jsonschema/analytics/mediawiki/ip_reputation/score/current.yaml

Mon, Dec 16, 11:52 AM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering
acooper added a comment to T381203: hCaptcha: Add a hook to modify score at runtime.

Is this where the request gets blocked if the score if >x? If so we don't need to do that next quarter.

Mon, Dec 16, 11:48 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)

Fri, Dec 13

Reedy added a comment to T382148: Create hcaptcha test wiki.

We can probably just create an instance under https://openstack-browser.toolforge.org/project/security-tools

Fri, Dec 13, 2:45 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper renamed T382151: Make hcaptcha compatible with NoJS browsers from Add NoJS support for hcaptcha to Make hcaptcha compatible with NoJS browsers.
Fri, Dec 13, 2:45 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
acooper assigned T382151: Make hcaptcha compatible with NoJS browsers to Reedy.
Fri, Dec 13, 2:44 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
acooper assigned T382150: Add hcaptcha support to API edits to Reedy.
Fri, Dec 13, 2:43 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
acooper updated the task description for T378194: hCaptcha: Implement no captcha mode (API/no js usage).
Fri, Dec 13, 2:42 PM · ConfirmEdit (CAPTCHA extension)
acooper updated the task description for T382148: Create hcaptcha test wiki.
Fri, Dec 13, 2:39 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T382148: Create hcaptcha test wiki.
Fri, Dec 13, 2:38 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T382148: Create hcaptcha test wiki.
Fri, Dec 13, 2:38 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper added a comment to T380625: Augment captcha logging.

Can defer this until spoken to the decision science team for advice

Fri, Dec 13, 2:32 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
Reedy added a project to T378188: Implement secure enclave mode for hCaptcha: FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:32 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
acooper added a project to T356599: DiscussionTools is incompatible with hCaptcha (and likely ReCaptcha): FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:31 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension), affects-Miraheze, DiscussionTools
acooper created T382148: Create hcaptcha test wiki.
Fri, Dec 13, 2:30 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T378194: hCaptcha: Implement no captcha mode (API/no js usage).
Fri, Dec 13, 2:29 PM · ConfirmEdit (CAPTCHA extension)
acooper updated the task description for T378194: hCaptcha: Implement no captcha mode (API/no js usage).
Fri, Dec 13, 2:28 PM · ConfirmEdit (CAPTCHA extension)
acooper added a project to T379179: Send captcha API response data to event logging: FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:27 PM · Data-Engineering-Radar, FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Metrics Platform, MediaWiki-extensions-EventLogging, ConfirmEdit (CAPTCHA extension), Data-Engineering
acooper added a project to T380625: Augment captcha logging: FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:27 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
acooper updated the task description for T378194: hCaptcha: Implement no captcha mode (API/no js usage).
Fri, Dec 13, 2:26 PM · ConfirmEdit (CAPTCHA extension)
acooper added a project to T378194: hCaptcha: Implement no captcha mode (API/no js usage): FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:24 PM · ConfirmEdit (CAPTCHA extension)
acooper added a project to T381203: hCaptcha: Add a hook to modify score at runtime: FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:23 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, ConfirmEdit (CAPTCHA extension)
acooper added a project to T379177: hCaptcha: Work out how to use returned score rather than just if the captcha was solved: FY2024-25 WE4.2.3 CAPTCHA evaluation framework.
Fri, Dec 13, 2:23 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Patch-For-Review, ConfirmEdit (CAPTCHA extension)

Dec 9 2024

Joe added a comment to T381265: hCaptcha IP blinding trial prototype / design.

hCaptcha provides some reference nginx implementations of option 3, which frankly looks like something we could easily implement as an extremely simple helm chart. Note that their example uses the remote IP as an identifier, which we would need to change, and amazingly generates a 128 bit hash, which is not what they suggest in the documentation (and that was leaving me perplexed).

Dec 9 2024, 11:12 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper added a comment to T381265: hCaptcha IP blinding trial prototype / design.

Looking at the nginx example from hcaptcha enterprise documentation the proxy works based on domain names rather than URL paths:

Dec 9 2024, 11:09 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework

Dec 2 2024

acooper updated the task description for T381265: hCaptcha IP blinding trial prototype / design.
Dec 2 2024, 12:13 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper created T381265: hCaptcha IP blinding trial prototype / design.
Dec 2 2024, 12:11 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework

Nov 28 2024

kostajh changed the status of T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha from Open to In Progress.
Nov 28 2024, 8:36 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team
kostajh changed the status of T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha, a subtask of T289607: <Security Initiative> Improving Captcha, from Open to In Progress.
Nov 28 2024, 8:36 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Foundational Technology Requests

Nov 21 2024

Jdforrester-WMF added a project to T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha: ConfirmEdit (CAPTCHA extension).
Nov 21 2024, 8:10 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework

Nov 13 2024

Reedy updated the task description for T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha.
Nov 13 2024, 5:34 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team
Reedy updated the task description for T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha.
Nov 13 2024, 5:25 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team
Reedy added a comment to T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha.

Noting also the description is rather out of date

Nov 13 2024, 5:24 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team

Oct 24 2024

Reedy moved T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha from Backlog to Feature Requests/Improvements on the ConfirmEdit (CAPTCHA extension) board.
Oct 24 2024, 1:32 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team
Reedy moved T333770: Evaluate Cloudflare Turnstile as alternative to FancyCaptcha at Wikimedia from Backlog to Feature Requests/Improvements on the ConfirmEdit (CAPTCHA extension) board.
Oct 24 2024, 1:27 AM · Accessibility, ConfirmEdit (CAPTCHA extension), Privacy
Reedy removed subtasks for T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha: T284894: MobileFrontend editor not compatible with ConfirmEdit/reCAPTCHA, T356599: DiscussionTools is incompatible with hCaptcha (and likely ReCaptcha).
Oct 24 2024, 1:25 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team
Reedy added a subtask for T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha: T284894: MobileFrontend editor not compatible with ConfirmEdit/reCAPTCHA.
Oct 24 2024, 1:15 AM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework, Software-Licensing, Tech-Product API Roadmap, Product Infrastructure Roadmap, Privacy, ConfirmEdit (CAPTCHA extension), Security-Team

Oct 18 2024

acooper updated the task description for T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha.
Oct 18 2024, 2:45 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha.
Oct 18 2024, 2:45 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha.
Oct 18 2024, 2:41 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha.
Oct 18 2024, 2:40 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper updated the task description for T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha.
Oct 18 2024, 2:39 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper created T377591: Minimum Viable Product to make ConfirmEdit work with hCaptcha.
Oct 18 2024, 2:38 PM · ConfirmEdit (CAPTCHA extension), FY2024-25 WE4.2.3 CAPTCHA evaluation framework
acooper closed T369816: Create a basic test harness for anti-captcha tool and run it as Resolved.
Oct 18 2024, 2:35 PM · FY2024-25 WE4.2.3 CAPTCHA evaluation framework