Broken download URLs #8149

c0dev0id · 2021-11-26T19:32:14Z

c0dev0id
Nov 26, 2021

Hey Github,

Something is broken with your download URLs. Previously working download URLs stopped working.
Can you please fix this? This happens for many ports connecting to github.

===> Returning to build of VVVVVV-2.3.4
===> VVVVVV-2.3.4 depends on: sdl2-mixer-* -> sdl2-mixer-2.0.4
===>  Verifying specs:  c   c  abi pthread SDL2 SDL2_mixer c m
===>  found c  .8.0 c  abi.5.1 pthread.26.1 SDL2.0.10 SDL2_mixer.0.1 c.96.1 m.10.1
===>  Checking files for VVVVVV-2.3.4
>> Fetch https://github.com/TerryCavanagh/VVVVVV/archive/2.3.4/VVVVVV-2.3.4.tar.gz
ftp: Error retrieving https://codeload.github.com/TerryCavanagh/VVVVVV/tar.gz/2.3.4/VVVVVV-2.3.4: 404 Not Found

Answered by gudmundur

Nov 28, 2021

Hi everyone. 👋🏻

A change in the handling of URL schemes was deployed a couple of days ago that caused the regression being discussed here. Due to the amount of traffic that the archive endpoints see, and the high baseline of 404s on them, this regression did not cause an unusual increase of errors that would've caused our alerting to kick in. The change has just been rolled back, so the issue is fixed. We will investigate this issue further after the weekend and take the appropriate steps to make sure similar regressions don't happen in the future.

Thank you for reaching out, your patience and understanding.

View full answer

sergiomb2 · 2021-11-26T23:12:57Z

sergiomb2
Nov 26, 2021

I also verify that download tags stooped to work .i.e.

wget https://github.com/sergiomb2/lpf/archive/v0.3/lpf-0.3.tar.gz stopped to work where v0.3 is a tag

but still work with a specific commit :
wget https://github.com/sergiomb2/lpf/archive/f9032ddd26ffb33fd140e0b9d5ee72c445608077/lpf-0.3.tar.gz

1 reply

ArchangeGabriel Nov 27, 2021

OP was also reporting the issue to be about tags, with a similarly constructed URL.

gotmax23 · 2021-11-27T03:21:58Z

gotmax23
Nov 27, 2021

This change has effectively broken every single Fedora Linux package that uses sources from Github1. I am sure that Fedora is far from the only entity effected by this issue. Please fix this as soon as possible.

Thanks.

2 replies

ArchangeGabriel Nov 27, 2021

Arch Linux as well for every package downloading tarballs from GitHub. Packages checking out the git source tree are not affected, but they are a minority.

xfgusta Nov 27, 2021

I was packaging pick when I faced this problem. I'm waiting for them to fix this.

https://github.com/mptre/pick/archive/v4.0.0/pick-4.0.0.tar.gz

davorpa · 2021-11-27T13:27:24Z

davorpa
Nov 27, 2021

Hi!!

My two cents. It's seems that latest releases download endpoint is broken too and it don't redirect well to related refs/tags

E.g.:

	URL
Endpoint:	https://github.com/microsoft/vscode/releases/latest/download/package.zip
Redirects to:	https://github.com/microsoft/vscode/releases/download/1.62.3/package.zip
Currently expected:	https://github.com/microsoft/vscode/archive/refs/tags/1.62.3.zip

Moreover, I'd be sure that the previous format about tags was [user]/[repo]/releases/tag/[version]/download/[filename] because I have some scripts to autoupdate some project binaries. Eg. jq https://github.com/stedolan/jq/releases/latest/download/jq-linux64 but it works

1 reply

hasokeric Nov 28, 2021

Great breakdown thanks

tinywrkb · 2021-11-28T12:51:39Z

tinywrkb
Nov 28, 2021

I suspect that this change will affect quite a few apps on Flathub, as it's an unofficial convention to use these download URLs that have the repo name as part of the filename, and the reason for it is better logging when the Flathub builder CI is automatically updating sources.

0 replies

StoneCypher · 2021-11-28T13:27:33Z

StoneCypher
Nov 28, 2021

I don't use releases much, so I may very well be misunderstanding this. Take what I'm saying with a large grain of salt.

I wanted to note that of the URLs that Davorpa put up, only the first two are broken. The third, which he lists as expected, actually works.

I looked at one of my projects which has releases set up. I tried recreating the .../releases/latest/download... url for it.

That sends it to my equivalent of his redirect URL, which fails as his does.

When I recreate his "expected" URL for my project, it works as his does. (Mine needs a v in front of the version number. It's not clear if that's meaningful or not.)

At this time I wonder whether either

The redirect is going to the wrong URL, or
Multiple URLs should be supported here but only some survive

The releases link on the repo webpage points to the Expected structure link, and works correctly.

I can't find a "how this is actually supposed to work" document, by which I could state clearly whether my repo is working as expected.

1 reply

davorpa Nov 28, 2021

Thanks for develop the proof ❤️. Definitely it is a breakdown, I though

The v doesn't matter since the url mapping follows the pattern: [user]/[repo]/releases/tag/[version]/....

version is the text what you want. A good practice is follow https://semver.org/ (without v)

gudmundur · 2021-11-28T16:16:20Z

gudmundur
Nov 28, 2021

Hi everyone. 👋🏻

A change in the handling of URL schemes was deployed a couple of days ago that caused the regression being discussed here. Due to the amount of traffic that the archive endpoints see, and the high baseline of 404s on them, this regression did not cause an unusual increase of errors that would've caused our alerting to kick in. The change has just been rolled back, so the issue is fixed. We will investigate this issue further after the weekend and take the appropriate steps to make sure similar regressions don't happen in the future.

Thank you for reaching out, your patience and understanding.

2 replies

mambon2 Jan 30, 2023

this is happening again today

thesamesam Jan 30, 2023

Looks intentional: https://github.blog/changelog/2023-01-30-git-archive-checksums-may-change/.

A rollout just for new archives would've helped a lot for distributions.

hsghn · 2022-11-21T12:47:26Z

hsghn
Nov 21, 2022

1

1 reply

hsghn Nov 21, 2022

Reply

jkeen · 2022-12-15T17:40:50Z

jkeen
Dec 15, 2022

@gudmundur I'm still running into this issue with an npm optional dependency pointing to organization/repo#sha. Whenever npm tries to install, it 404s trying to resolve codeload.github.com/organization/repo/tar.gz/sha. I have access to the organization, and can download the tar successfully at github.com/organization/repo/archive/sha.tar.gz

How can I get around this?

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Community

Broken download URLs #8149

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 8 comments 8 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Broken download URLs #8149

Replies: 8 comments · 8 replies

Replies: 8 comments 8 replies