Adding protocol handlers #972

diekus · 2021-04-26T09:37:33Z

This is an update that brings the protocol_handlers member addition up to date with the new structure of the specification (as stated in #969, it has been reworked since the previous effort was based on a very old version of the manifest spec).

cc @fabiorocha @aarongustafson

This change (choose at least one, delete ones that don't apply):

Adds new normative requirements
Adds new normative recommendations or optional items

Implementation commitment (delete if not making normative changes):

If change is normative, and it adds or changes a member:

[updated JSON

Person merging, please make sure that commits are squashed with one of the following as a commit message prefix:

chore:
editorial:
BREAKING CHANGE:
And use none if it's a normative change

Preview | Diff

Adds protocol_handlers member and protocolHandlerItem.

Adds protocol_handlers member to manifest file.

index.html

-removes return statements that exited the loop while processing protocol_handlers.

- Adds for each to its infra definition.

index.html

marcoscaceres

This is a good start, but it needs a lot tighter integration with registerProtocolHandler on the security side - they should both use the same underlying model, and I'm not getting a sense this does. The URL replacement parts are also quite scary - all this should go through HTML's "normalize protocol handler parameters". The examples already show how this would be open to abuse (e.g., using random protocols shouldn't work - as that open up a significant attack avenue).

index.html

Co-authored-by: Marcos Cáceres <[email protected]>

aarongustafson · 2021-05-13T20:42:59Z

This is a good start, but it needs a lot tighter integration with registerProtocolHandler on the security side - they should both use the same underlying model, and I'm not getting a sense this does. The URL replacement parts are also quite scary - all this should go through HTML's "normalize protocol handler parameters".

@marcoscaceres In line 1070, @diekus’ proposal punts to the HTML spec for the process of registering a protocol handler (see "register a protocol handler"), which is where all of the normalization and security stuff takes place. Do you think that needs to be framed differently to make that connection more clear for implementors?

        <p>
          To process the the <var>processedProtocolHandlers</var> the user
          agent SHOULD [=register a protocol handler=] per item defined in the
          [=sequence=].
        </p>

The examples already show how this would be open to abuse (e.g., using random protocols shouldn't work - as that open up a significant attack avenue).

Perhaps I’m misreading your concern here, but the second bullet in Example 6 specifically calls out that attempted abuse (like attempting to associate with "store") would be thrown out. Am I missing something?

            <li>The second protocol handler would be ignored, as the protocol
            provided does not start with "web " and is not part of the
            safelist.
            </li>

updates spec to remove traces of the old WebIDL ways.

…ifest into adding-protocol-handlers

index.html

marcoscaceres · 2021-05-18T08:07:17Z

index.html

+ The user agent MUST ask for permission when using a protocol
+ handler for the first time. This feature requires user interaction
+ and a script cannot communicate with another application on its


This seems odd (not the asking for permission part - but this doesn't feel like it's wired up properly)...

Using HTML JS as an example, can you show me what would this look like in practice? If it's to do with "permissions", then I suspect we need some kind of Permissions Policy or Permissions API thing here.

Hola Marcos, so the way this would work is that these registrations happen upon installation of the Web App. There is a prompt, and it is similar to the one that appears when registration is invoked from the .registerProtocolHandler(). In this case, where the registration happens from the manifest, the prompt appears the first time the protocol handler is invoked for a Web app. (It was a requirement from security/privacy folks).

In this case, where the registration happens from the manifest, the prompt appears the first time the protocol handler is invoked for a Web app.

Sorry, I'm still having a hard time following. Can you walk me through the set up end-to-end?

How do I "invoke a protocol handler"? Do I click on a link on some website (appears to be the case from the HTML spec)?

I'm also confused about this in the HTML spec:

navigator.registerProtocolHandler('web soup', 'soup?url=%s')

If I can register protocol handlers to do the same thing in HTML using JS, why do we need them in the manifest again? Does this just duplicate the functionality? Or does this work differently or provide some additional functionality that I'm not getting?

Sorry for the dumb questions. I'm just trying to understand.

Hola again Marcos! Not sure which dumb questions you are referring to, they all seem important to be clear for me!

The difference between navigator.registerProtocolHandler() and having a protocol_handlers member in the manifest file is is mostly where/when is the scheme registration and invocation happening. Let me explain a bit further:

navigator.registerProtocolHandler(): once it is called it will immediately try to register the scheme and show the prompt displayed above. This is handy if I want to register a scheme anytime from HTML/JS. Handy AF.

protocol_handlers member in the manifest file: this will register a/several scheme(s) when a Web application is being installed. (register the installed Web app as a possible handler for that scheme). Here a noteworthy difference is that the prompt itself would appear once a user in a website on the browser invokes one of the registered schemes. For example, I installed the BatmanPWA and when I am browsing online I click on a 'web batman://' link and then the prompt appears in the browser asking if the user wants to use that (registered) PWA. This happens the first time a user invokes a specific scheme. Very convenient indeed.

So in summary, it is a matter of where the registration happens, as it is more convenient for a user to decide whether they want to use an installed Web app when they run into a link that would require it instead of getting N prompts when trying to install a Web app (where N = number of valid items in the protocol_handlers member in the manifest).

There is also the automatic unregistration that happens when a user uninstalls a webapp, which is nice :)

protocol_handlers member in the manifest file: this will register a/several scheme(s) when a Web application is being installed.

Ok, but isn't this in violation of the web's security/permission model? I.e., we only really allow things to be request permission at use time, not en mass at install time.

Consider, we've previously rejected similar proposals whereby one would do:

features: ["geolocation", "camera", "something else"]

This feels very similar.

Here a noteworthy difference is that the prompt itself would appear once a user in a website on the browser invokes one of the registered schemes. For example, I installed the BatmanPWA and when I am browsing online I click on a 'web batman://' link and then the prompt appears in the browser asking if the user wants to use that (registered) PWA. This happens the first time a user invokes a specific scheme. Very convenient indeed.

This is concerning though. If I install BatmanPWA, it could do:

protocol_handlers: [ /// ...1000 protocols here... MUAHAHAH! /// Appear to handle everything, and show up everywhere! ]

@dmurph:

There is also the automatic unregistration that happens when a user uninstalls a webapp, which is nice :)

Agree, that's nice. But it does presume some tight integration at the OS level with the browser. IIRC (and this was long ago), Firefox couldn't support such things because it could only put a shortcut icon on the home screen, but it had no way of getting notified if a user had deleted the shortcut.

Hola Marcos,

protocol_handlers member in the manifest file: this will register a/several scheme(s) when a Web application is being installed.

Ok, but isn't this in violation of the web's security/permission model? I.e., we only really allow things to be request permission at use time, not en mass at install time.

It is as you are saying, the permission request happens when the user is using the protocol for the first time. We think the consent dialog on this first use is a good compromise between asking for permission for every protocol that an app wants to register and the security concerns of registering the protocols.

Consider, we've previously rejected similar proposals whereby one would do:

features: ["geolocation", "camera", "something else"]

This feels very similar.

Here a noteworthy difference is that the prompt itself would appear once a user in a website on the browser invokes one of the registered schemes. For example, I installed the BatmanPWA and when I am browsing online I click on a 'web batman://' link and then the prompt appears in the browser asking if the user wants to use that (registered) PWA. This happens the first time a user invokes a specific scheme. Very convenient indeed.

This is concerning though. If I install BatmanPWA, it could do:

protocol_handlers: [ /// ...1000 protocols here... MUAHAHAH! /// Appear to handle everything, and show up everywhere! ]

The list of safe-listed protocols is rather small and they're very likely to have multiple handlers already defined among browser, native and installed web apps, which makes it unlikely that an app can take over everything. Would setting an arbitrary limit to the number of protocols that can be registered help?

@dmurph:

There is also the automatic unregistration that happens when a user uninstalls a webapp, which is nice :)

Agree, that's nice. But it does presume some tight integration at the OS level with the browser. IIRC (and this was long ago), Firefox couldn't support such things because it could only put a shortcut icon on the home screen, but it had no way of getting notified if a user had deleted the shortcut.

index.html

Co-authored-by: Marcos Cáceres <[email protected]>

diekus · 2021-06-07T20:58:48Z

@aarongustafson wrote:

@marcoscaceres In line 1070, @diekus’ proposal punts to the HTML spec for the process of registering a protocol handler (see "register a protocol handler"), which is where all of the normalization and security stuff takes place. Do you think that needs to be framed differently to make that connection more clear for implementors?

Yes. At at minimum, we need to deal with the errors thrown by "register a protocol handler" (that should be simple enough... "if it throws, continue".). Also, the "register a protocol handler" takes "normalizedScheme and normalizedURLString", which is normalization we don't do.

These above small things, but I want to make sure the integration is done as expected. Right now, it feels like we are not taking as much care as we should in passing the right things to HTML.

I have changed the processing to better align (uses the same normalization for scheme and url parameters as rph())

diekus · 2021-06-14T17:49:53Z

friendly ping @marcoscaceres 👋🏼

marcoscaceres · 2021-06-15T08:25:38Z

Hi @diekus, thanks for continuing to work on this. I've update the initial comment to include the things that are missing still for us to proceed. In particular, we need backing from a second engine and no objections.

index.html

marcoscaceres · 2021-06-15T08:47:02Z

Moving to general discussion here...

Would setting an arbitrary limit to the number of protocols that can be registered help?

It feels like it would still be potentially overwhelming.

This is why registration via the existing APIs seems favorable to me. It already supports arbitrary same-origin URL registrations with good error handling?

So, fundamental question is: what significant value does this add over the existing protocol handler API? Also, if the PWA is not installed, won't developers need to call .registerProtocolHandler() anyway to support users who don't want the app on their home screen, but still want protocol handler capabilities?

Co-authored-by: Marcos Cáceres <[email protected]>

Helps with w3c/manifest#972.

diekus · 2021-06-28T10:54:02Z

Hola @marcoscaceres.

So, fundamental question is: what significant value does this add over the existing protocol handler API?

We think it makes a lot of sense to allow a developer to register protocol handlers in the manifest. It is where they will also register file handlers, link handlers and share targets, all in a way that ties them up nicely with the application itself.

Additionally, it is appropriate because they are linked to the app lifecycle, providing deregistration of those schemes once the app is uninstalled (unlike registerProtocolHandler()).

From the UX side, if an application needed to register to handle N protocols upon installing, it's bad UX to prompt the user N times for the permissions. With the protocols installed from the manifest, the user would only get a permission prompt the first time they link on a link.

Finally, we think it's about the constant evolution in HTML to make things more declarative. With the protocol_handlers member in the manifest, you don't need to rely on JS to register schemes for your installed app.

Also, if the PWA is not installed, won't developers need to call .registerProtocolHandler() anyway to support users who don't want the app on their home screen, but still want protocol handler capabilities?

Yes, but both are valid. I may want to have both the "mailclient.com" in the browser and additionally have the "Mail Client PWA" registered as a handler for mailto://. (one through registerProtocolHandler() and the other one through the manifest).

marcoscaceres · 2021-06-30T03:31:31Z

We think it makes a lot of sense to allow a developer to register protocol handlers in the manifest. It is where they will also register file handlers, link handlers and share targets, all in a way that ties them up nicely with the application itself.

This makes sense, particularly for new types (e.g., share target). There's been hesitancy amongst implementers in the past to duplicate functionality already provided by a JS API. Having said that...

Additionally, it is appropriate because they are linked to the app lifecycle, providing deregistration of those schemes once the app is uninstalled (unlike registerProtocolHandler()).

That's compelling, indeed - although the site could call unregisterProtocolHandler(). I guess this is interesting if protocols are "provisionally registered", but are inert until activated.

As an aside, we probably want to be clear with what happens when .unregisterProtocolHandler() is called, if it's not already.

From the UX side, if an application needed to register to handle N protocols upon installing, it's bad UX to prompt the user N times for the permissions. With the protocols installed from the manifest, the user would only get a permission prompt the first time they link on a link.

And this would only be for that particular link type, right? Not a blanket approval for all registered things?

Finally, we think it's about the constant evolution in HTML to make things more declarative. With the protocol_handlers member in the manifest, you don't need to rely on JS to register schemes for your installed app.

There are a lot of pros/cons here (personally, I like the imperative APIs over the declarative ones) - but let's keep this outside.

marcoscaceres · 2021-06-30T03:38:19Z

@diekus, could you please run HTM5 tidy on PR? There seem to be a couple of tiny markup issues:

line 1035 column 13 - Warning: missing </ol> before </li>
line 1052 column 11 - Warning: discarding unexpected </li>
line 1011 column 7 - Warning: missing </section> before <li>
line 1053 column 11 - Warning: inserting implicit <ul>
line 1056 column 7 - Warning: discarding unexpected </ol>
line 1053 column 11 - Warning: missing </ul> before </section>

run tidy on manifest draft

runs tidy with the correct tidy config file 👀👀

diekus · 2021-07-05T18:40:18Z

We think it makes a lot of sense to allow a developer to register protocol handlers in the manifest. It is where they will also register file handlers, link handlers and share targets, all in a way that ties them up nicely with the application itself.

This makes sense, particularly for new types (e.g., share target). There's been hesitancy amongst implementers in the past to duplicate functionality already provided by a JS API. Having said that...

Additionally, it is appropriate because they are linked to the app lifecycle, providing deregistration of those schemes once the app is uninstalled (unlike registerProtocolHandler()).

That's compelling, indeed - although the site could call unregisterProtocolHandler(). I guess this is interesting if protocols are "provisionally registered", but are inert until activated.

As an aside, we probably want to be clear with what happens when .unregisterProtocolHandler() is called, if it's not already.

So registering protocol handlers from the manifest file would register them for the installed Web app. Registering protocol handlers through registerProtocolHandler() would register the website through the browser, therefore calling unregisterProtocolHandler() would not unregister any handlers from installed Web apps. In a way, the two ways of registering handlers are different because they act on different scopes.

From the UX side, if an application needed to register to handle N protocols upon installing, it's bad UX to prompt the user N times for the permissions. With the protocols installed from the manifest, the user would only get a permission prompt the first time they link on a link.

And this would only be for that particular link type, right? Not a blanket approval for all registered things?

Correct, it would be only for the particular link type that is triggered. Security wise, every different protocol registration needs to be granted permission on first use of that particular link type.

Finally, we think it's about the constant evolution in HTML to make things more declarative. With the protocol_handlers member in the manifest, you don't need to rely on JS to register schemes for your installed app.

There are a lot of pros/cons here (personally, I like the imperative APIs over the declarative ones) - but let's keep this outside.

adds definition of safelisted schemes

lauramorinigo · 2021-07-26T13:17:09Z

1 to this initiative, it can be a good start point for other similar cases like RSS.

mgiuca

Some drive-by comments.

mgiuca · 2021-07-28T08:09:52Z

index.html

+ <ol>
+ <li>Let <var>processedProtocolHandlers</var> be a new [=list=].
+ </li>
+ <li>[=list/For each=] <var>protocol_handler</var> (<a>protocol


Be consistent with variable naming. Looks like we're using camelCase, so protocolHandler not protocol_handler.

Question for @kenchris : did this change recently? I'm looking at the shortcuts algorithm above which uses variables like processedShortcuts. I swear we used to use variable names with spaces in them, not camelCaseVariableNames.

index.html

mgiuca · 2021-07-28T08:20:16Z

index.html

+ </li>
+ </ol>
+ <ul>
+ <li>[=list/For each=] |processedProtocolHandlers|, [=register a


I don't think this can appear here without context (not part of any algorithm). It doesn't say when it should happen, or what the requirement level is (I think it should be a SHOULD; without that it implies MUST).

At a minimum, this should be in an algorithm "To register the protocol handlers for a processed manifest manifest", then you can properly index into that manifest in the for loop, e.g. "For each protocolHandler of manifest["protocol_handlers"]".

Ideally, this would be called from an algorithm that's invoked at install time, but I don't see one of those. So maybe just add a note to the "Installable web applications" section to say the user agent MAY [=register the protocol handlers=]".

I have added the SHOULD requirement level for the registration and fixed the loose ul element that made it appear as orphan. I have also added a note to the installable web applications section indicating that a protocol handler may be installed.

mgiuca · 2021-07-28T08:24:19Z

index.html

+ </section>
+ <section>
+ <h3>
+ Privacy and Security considerations


I'm not sure this should go here. There's a main P&S section at the end of the document that collects all of these (this would be the only section to have it's own P&S subsection, other than Installable Web Apps, and that is probably a mistake too -- note that it is duplicated text from the end of the document).

However, I think P&S should only have non-normative discussion; a MUST requirement shouldn't be in it. So I would move this MUST requirement above to Handling a protocol launch. Then explain why it is so in the P&S section.

mgiuca · 2021-07-28T08:25:27Z

index.html

+ </p>
+ <p>
+ The user agent MUST ask for permission when using a protocol handler
+ for the first time. This feature requires user interaction and a


I don't understand this second sentence. Are you trying to write a normative requirement that registration requires user interaction? Are you justifying the previous MUST requirement? Please rephrase this.

index.html

- fixes orphan ul element - moves protocol handler privacy and security section to main P&S section

Helps with w3c/manifest#972.

diekus added 2 commits April 15, 2021 15:19

adds protocol_handlers member

999f9a9

Adds protocol_handlers member and protocolHandlerItem.

Update index.mhtml (adds protocol_handlers)

da610bd

Adds protocol_handlers member to manifest file.

diekus requested a review from marcoscaceres April 26, 2021 09:37

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

diekus added 2 commits April 26, 2021 12:40

update index.htmlo

8664c01

-removes return statements that exited the loop while processing protocol_handlers.

Update index.html

228e2e6

- Adds for each to its infra definition.

diekus requested a review from kenchris April 26, 2021 11:51

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

kenchris reviewed Apr 26, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

marcoscaceres requested changes Apr 27, 2021

View reviewed changes

diekus and others added 3 commits May 7, 2021 12:23

Update index.html

e1b678f

Co-authored-by: Marcos Cáceres <[email protected]>

Update index.html

81bdeda

Co-authored-by: Marcos Cáceres <[email protected]>

Update index.html

2c023ef

Co-authored-by: Marcos Cáceres <[email protected]>

Schweinepriester mentioned this pull request May 11, 2021

Add URL Protocol Handler Registration for PWAs Fyrd/caniuse#5900

Open

Merge branch 'gh-pages' into adding-protocol-handlers

2ace6ea

diekus added 2 commits May 17, 2021 18:46

updates to protocol_handlers

61780a4

updates spec to remove traces of the old WebIDL ways.

Merge branch 'adding-protocol-handlers' of https://github.com/w3c/man…

8b698d5

…ifest into adding-protocol-handlers

diekus requested review from kenchris and marcoscaceres May 17, 2021 19:46

Update index.html

9314353

marcoscaceres reviewed May 18, 2021

View reviewed changes

marcoscaceres requested changes May 18, 2021

View reviewed changes

diekus and others added 3 commits May 18, 2021 09:58

Update index.html

264ab23

Co-authored-by: Marcos Cáceres <[email protected]>

Update index.html

b011275

Co-authored-by: Marcos Cáceres <[email protected]>

Update index.html

1dcffe3

Co-authored-by: Marcos Cáceres <[email protected]>

Merge branch 'gh-pages' into adding-protocol-handlers

ea5cb7b

diekus requested a review from marcoscaceres June 7, 2021 21:40

marcoscaceres reviewed Jun 15, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

marcoscaceres reviewed Jun 15, 2021

View reviewed changes

index.html Outdated Show resolved Hide resolved

diekus and others added 2 commits June 15, 2021 14:38

Update index.html

2573589

Co-authored-by: Marcos Cáceres <[email protected]>

Merge branch 'gh-pages' into adding-protocol-handlers

3998cfa

domenic pushed a commit to whatwg/html that referenced this pull request Jun 25, 2021

Meta: export "normalize protocol handler parameters"

a6e18b8

Helps with w3c/manifest#972.

diekus requested a review from marcoscaceres June 28, 2021 10:54

Merge branch 'gh-pages' into adding-protocol-handlers

8308ae2

marcoscaceres mentioned this pull request Jul 1, 2021

URL Protocol Handler Registration for web apps mozilla/standards-positions#340

Open

diekus added 2 commits July 5, 2021 15:13

Run tidy

2b4ba13

run tidy on manifest draft

run tidy

c034185

runs tidy with the correct tidy config file 👀👀

diekus and others added 3 commits July 12, 2021 19:27

Merge branch 'gh-pages' into adding-protocol-handlers

40f8a13

Update index.html

73f3812

adds definition of safelisted schemes

Merge branch 'gh-pages' into adding-protocol-handlers

ec620b6

mgiuca reviewed Jul 28, 2021

View reviewed changes

diekus and others added 2 commits August 2, 2021 20:06

Fixes index.html

6de904e

- fixes orphan ul element - moves protocol handler privacy and security section to main P&S section

Merge branch 'gh-pages' into adding-protocol-handlers

ee2674a

filips123 mentioned this pull request Jan 1, 2022

Support for protocol handlers (mailto, etc.) filips123/PWAsForFirefox#83

Closed

mfreed7 pushed a commit to mfreed7/html that referenced this pull request Jun 3, 2022

Meta: export "normalize protocol handler parameters"

551a515

Helps with w3c/manifest#972.

Merge branch 'main' into adding-protocol-handlers

b8c938a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding protocol handlers #972

Adding protocol handlers #972

diekus commented Apr 26, 2021 •

edited by pr-preview bot

Loading

marcoscaceres left a comment

aarongustafson commented May 13, 2021

marcoscaceres May 18, 2021

diekus May 24, 2021

marcoscaceres May 25, 2021 •

edited

Loading

diekus May 25, 2021 •

edited

Loading

dmurph May 26, 2021

marcoscaceres May 28, 2021

diekus Jun 7, 2021

diekus commented Jun 7, 2021

diekus commented Jun 14, 2021

marcoscaceres commented Jun 15, 2021 •

edited

Loading

marcoscaceres commented Jun 15, 2021

diekus commented Jun 28, 2021

marcoscaceres commented Jun 30, 2021

marcoscaceres commented Jun 30, 2021 •

edited

Loading

diekus commented Jul 5, 2021

lauramorinigo commented Jul 26, 2021

mgiuca left a comment

mgiuca Jul 28, 2021

mgiuca Jul 28, 2021

diekus Aug 2, 2021

mgiuca Jul 28, 2021

mgiuca Jul 28, 2021

Adding protocol handlers #972

Are you sure you want to change the base?

Adding protocol handlers #972

Conversation

diekus commented Apr 26, 2021 • edited by pr-preview bot Loading

marcoscaceres left a comment

Choose a reason for hiding this comment

aarongustafson commented May 13, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcoscaceres May 25, 2021 • edited Loading

Choose a reason for hiding this comment

diekus May 25, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diekus commented Jun 7, 2021

diekus commented Jun 14, 2021

marcoscaceres commented Jun 15, 2021 • edited Loading

marcoscaceres commented Jun 15, 2021

diekus commented Jun 28, 2021

marcoscaceres commented Jun 30, 2021

marcoscaceres commented Jun 30, 2021 • edited Loading

diekus commented Jul 5, 2021

lauramorinigo commented Jul 26, 2021

mgiuca left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diekus commented Apr 26, 2021 •

edited by pr-preview bot

Loading

marcoscaceres May 25, 2021 •

edited

Loading

diekus May 25, 2021 •

edited

Loading

marcoscaceres commented Jun 15, 2021 •

edited

Loading

marcoscaceres commented Jun 30, 2021 •

edited

Loading