Page MenuHomePhabricator

ExtensionDistributor: Some extensions have no versions available for download
Closed, ResolvedPublic

Description

Since a few days, there are reports of the version dropdown being empty even though the branches exist. This makes it impossible to download extensions from ExtensionDistributor.

https://www.mediawiki.org/wiki/Thread:Extension_talk:CirrusSearch/CirrusSearch_download
https://www.mediawiki.org/wiki/Thread:Project:Current_issues/Can't_download_extensions


Version: master
Severity: major

Details

Reference
bz65852

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:11 AM
bzimport set Reference to bz65852.

-> https://www.mediawiki.org/wiki/Special:ExtensionDistributor/CirrusSearch
I see a master and 1.22 branch :/

-> https://www.mediawiki.org/wiki/Special:ExtensionDistributor?extdist_extension=AbuseFilter
Abuse Filter all branches up from 1.19 :) Tested with Firefox 17.0.1 ESR and Google Chrome 34.0.1847.116m

I see 1.22 and master for CirrusSearch. Giving me https://codeload.github.com/wikimedia/mediawiki-extensions-CirrusSearch/legacy.tar.gz/REL1_22 to download

There is no 1.23, but that isn't setup in CommonSettings.php

https://www.mediawiki.org/wiki/Special:ExtensionDistributor/MaintenanceShell (original report) does have empty dropdown for me, how common is this? How are they picked?

Problem does also exist for downloading Oversight (is that extension even downloadable?), see https://www.mediawiki.org/wiki/Special:ExtensionDistributor/Oversight. Status can be changed to CONFIRMED.

Created attachment 15498
branches exist

Hmm, anything i do false :?

Now i have tested with FF, Chrome and Android/HTC Stock Browser

Attached:

image:22087 (1×1 px, 128 KB)

Hmm, now i have now list of versions, too, on some Extensions :/

I've checked all extensions linked in comments from this bug and all of them list 1.23 and older in their dropdown. It's still an issue?

Note that some extensions weren't properly branched for 1.23, downloading an old (and probably incompatible) version, but that's bug 64157

(In reply to Florian from comment #8)

One example that doesn't work:

https://www.mediawiki.org/wiki/Special:ExtensionDistributor/GoogleLogin

That one has an explanation. It doesn't have any REL1_XX branch [1]

https://git.wikimedia.org/branches/mediawiki/extensions/GoogleLogin.git

ED shouldn't be listed in the extension's page

Oh, yeah, i thought there was one, will create it, thanks :) i will look into some extensions i remember tomorrow, when i'm at pc again.

(In reply to Florian from comment #11)

One more example:
https://www.mediawiki.org/wiki/Special:ExtensionDistributor/
ExtensionDistributor

(It's ExtensionDistributor itself)

Works for me at this moment

Hmm, now it works for me, too :/

Works here also, couldn't find any extension where ExtensionDistributor fails. Btw, is that URL in this bugs' settings really needed?

Created attachment 15932
Example extension Nonlinear

https://www.mediawiki.org/wiki/Special:ExtensionDistributor/Nonlinear

This extension actually doesn't work for me.

Attached:

Nonlinear.png (253×1 px, 16 KB)

Seems to be somewhat random. Versions appearing and disappearing regardless of their branches existing.

Are we being hit by github's ratelimiting? (I don't know if we have a special exemption or anything...)

Via eval.php:

$httpOptions = array( 'followRedirects' => false );

$httpOptions['proxy'] = $wgExtDistProxy;

$req = MWHttpRequest::factory("https://api.github.com/rate_limit", $httpOptions);

var_dump($req->getContent());

string(177) "{"resources":{"core":{"limit":60,"remaining":0,"reset":1405398906},"search":{"limit":10,"remaining":10,"reset":1405396979}},"rate":{"limit":60,"remaining":0,"reset":1405398906}}"

legoktm@terbium:~$ curl --proxy url-downloader.wikimedia.org:8080 -i https://api.github.com/repos/wikimedia/mediawiki-extensions-MassMessage/tarball/REL1_23

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0

and the content body:

{

"message": "API rate limit exceeded for <ip>. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
"documentation_url": "https://developer.github.com/v3/#rate-limiting"

}

Change 146386 had a related patch set uploaded by Legoktm:
Allow making API requests as an authenticated user

https://gerrit.wikimedia.org/r/146386

Change 146394 had a related patch set uploaded by Florianschmidtwelzow:
Display an error if no branches are found

https://gerrit.wikimedia.org/r/146394

Change 146394 merged by jenkins-bot:
Display an error if no branches are found

https://gerrit.wikimedia.org/r/146394

Change 146386 merged by jenkins-bot:
Allow making API requests as an authenticated user

https://gerrit.wikimedia.org/r/146386

I203d635b2a5e42197913d7d8b7cd8da083dd8981 was also merged, which will provide negative cache entries when a branch doesn't exist, which should make this a lot better in the short term.

After discussing with ^d and James_F, we're going to stop using github and instead use a labs setup I created (ex: http://extdist.wmflabs.org/?version=REL1_23&ext=VisualEditor) which will make it so we don't need to worry about ratelimiting at all.

We're no longer using github at all, so this should no longer be an issue.