Page MenuHomePhabricator

Support inline links to audio with ability to play it in situ
Closed, ResolvedPublicFeature

Assigned To
None
Authored By
Nardog
Jul 27 2019, 9:12 PM
Referenced Files
None
Tokens
"Love" token, awarded by Bugreporter2."Love" token, awarded by alexhollender_WMF."Heartbreak" token, awarded by Sdkb."Love" token, awarded by lionel.rowe."Love" token, awarded by MichaelSchoenitzer.

Description

For referencing audio files inline, such as pronunciation demonstrations, Wikipedia has relied on linking to the raw file using [[Media:...]], via templates like {{Audio}}.

But not all browsers support playing the linked file (which can be Ogg Vorbis, WAV, FLAC, WebM, Opus, or MIDI), causing them to download the file instead of playing it, even though the servers automatically transcode audio files into Ogg and MP3 specifically so that browsers can play them.

And even when the browser supports it, it is not user-friendly to suddenly send them to a different page with nothing but a player on it (a violation of the principle of least astonishment).


Luckily we have HTML5 and <audio> now, so we can let browsers figure out which format to use without much hassle.
I'm wondering if something like the following is feasible:

<audio file="Example.ogg">Link</audio>

in wikitext will fetch the derivative URLs and turn into something like:

<a href="/wiki/File:Example.ogg" title="Example.ogg" data-audiolink="[{&quot;src&quot;:&quot;//upload.wikimedia.org/wikipedia/commons/c/c8/Example.ogg&quot;,&quot;type&quot;:&quot;audio/ogg&quot;},{&quot;src&quot;:&quot;//upload.wikimedia.org/wikipedia/commons/transcoded/c/c8/Example.ogg/Example.ogg.mp3&quot;,&quot;type&quot;:&quot;audio/mpeg&quot;}]">Link</a>

and a JS will attach a handler that converts the JSON data to an HTMLAudioElement and plays it to the link's click event.

Event Timeline

I've written a user script that effectively achieves this. Still it would be nice if it was supported natively in wiki syntax, perhaps via a MW extension.

This task was mentioned at https://en.wikipedia.org/wiki/Template_talk:Audio#Clicking_shouldn't_open_up_an_entirely_new_page. I hope it gets some attention and doesn't just end up stuck in the backlog forever.

What is "situ" (mentioned in the task title)?

lionel.rowe rescinded a token.
lionel.rowe awarded a token.
lionel.rowe subscribed.

This would be a massive improvement to pages such as https://en.wikipedia.org/wiki/Help:IPA, with a large number of very short audio clips that you'd often want to replay or play back-to-back to compare sounds.

Here's another PoC user script I wrote, somewhat more minimal than @Nardog 's above (no additional UI):

https://en.wikipedia.org/wiki/User:Lionel.rowe/inline-audio.js

It checks for e.ctrlKey, so you can still choose to open in a new tab if you want.

TheDJ changed the subtype of this task from "Task" to "Feature Request".

So your proposal is to turn [[Media:]] links into html5 players right ? But that requires rendering, you can only do that for [[File:]] transcluding. At most for Media links you can do:

[[Media:example.ogg]]

<a href="downloadlink">link</a>

then enhance with JS click to play handler based on the filename in the download link. I'm not a big fan of this, as I do not consider those urls to be a stable api, and it requires guessing based on file extension that something is audio/video. I really don't like it.

So I think these are better options.

[[File:example.ogg|link|caption]] or maybe [[File:example.ogg|noplayer|caption]] (noplayer is an old option we might be able to reintroduce)

<a href="downloadlink" data-player="<audio or video src='playbacksrc'></audio or video>">caption</a>

then enhance with JS click to play handler, opening in the dialog we have now. Or

[[File:example.ogg|link|caption]] or maybe [[File:example.ogg|noplayer|caption]] (noplayer is an old option we might be able to reintroduce)

<span class="mw-tmh-playback-element"><audio or video src="playbacksrc"></audio or video><a class="mw-tmh-playback-link">caption</a></span>

where the player is hidden by default with CSS, and the link is enhanced with a JS click to play handler. This might be better for certain 3rd parties where no JS or CSS is used.. but I think that is pretty rare, so I'm not a big fan.

All have a problem however.. and that is with parsoid/VE. We'd probably have to introduce some sort of new element to Parsoid to allow for back and forth serialisation between html and wikitext without losing data.....

The wiki syntax can be anything; what I want is for the server to provide the derivative URLs (and MIME types) so they won't have to be fetched client-side. Firefox can't play WAVs, no browser can MIDIs... that's a bigger issue with [[Media:...]] than that you're led to a different page.

@cscott i'm wondering what parsing team thinks of this.. I basically need completely different HTML output for a certain version of a usage of an audio/video file.

I'm assuming that with the way Parsoid is currently written that it won't easily know about that and it likely also makes it harder for VE etc? Anyway before I introduce a new structure, i'd rather discuss this with you guys.

Why do you need "different HTML output"? As in the task description I've always suggested an extension tag. If the tag only supports audio, then it can just throw an error when it's given non-audio (or non-AV).

"turn [[Media:]] links into html5 players" indeed doesn't sound like a good idea and I'm at a complete loss as to how you thought I was suggesting that.

Why do you need "different HTML output"? As in the task description I've always suggested an extension tag. If the tag only supports audio, then it can just throw an error when it's given non-audio (or non-AV).

That just changes the problem into "reimplement everything for the File syntax in yet another hook for both the old parser and parsoid'. Remember that parsoid is bidirectional, so whatever html you generate from the wikitext, it needs to be able to turn it back in the exact same original wikitext (including all optional elements).

This (we hope) will effectively be solved by MediaWiki-extensions-Phonos. Here's an example page: https://en.wikipedia.beta.wmflabs.org/wiki/Phonos_without_any_IPA

Does that satisfy the wish? If not, what is missing? Phonos creates an Audio object in JavaScript and plays that, similar to what is being proposed. For no-JS users, it is simply a link to the file which may or may not force a download, but that is a small-ish percentage of users.

w:Template:Audio and similar templates can easily be converted to use Phonos. The one side effect I suppose is that the play icon is also provided by Phonos, so you won't have whatever designs the template uses.

@MusikAnimal note.. if you link to an oga, that doesn't work on iOS/macOS, as there is no native codec support. Either it would have to be smart and lookup the .mp3 derivative that TMH has created, or you would have to integrate with videojs. I only just realised this yesterday.

Yeah, it should definitely fetch derivative URLs and create an HTMLSourceElement for each.

@MusikAnimal note.. if you link to an oga, that doesn't work on iOS/macOS, as there is no native codec support. Either it would have to be smart and lookup the .mp3 derivative that TMH has created, or you would have to integrate with videojs. I only just realised this yesterday.

Okay, I'm glad I asked! I have filed T322368. Let's do any follow-up discussion there, in the event Phonos isn't the answer to the wish described here (but I hope it will be :)

Hello! The Wishlist survey says that this is "in development", but it doesn't seem to be any update in 6 months. Is it really in development? Can it be tested somehow? Thanks.

Great! Is there any idea on when will be deployed?

Great! Is there any idea on when will be deployed?

Apologies, I didn't see this — I believe I'm correct in saying "soon", and I may be able to provide more concrete timelines after a meeting next Thursday (8th June 2023).

T336763: Enable PhonosInlineAudioPlayerMode on all projects related.

Great! Is there any idea on when will be deployed?

Apologies, I didn't see this — I believe I'm correct in saying "soon", and I may be able to provide more concrete timelines after a meeting next Thursday (8th June 2023).

T336763: Enable PhonosInlineAudioPlayerMode on all projects related.

Following up, though not with a directly concrete answer — the inline audio player will be deployed to all projects, after the deployment order has been decided. It'll be soon 🙂