The generic "info" icon in the screenshots above is temporary until we come up with something better. @Pginer-WMF: any ideas for an icon for this notification? @Cenarium suggested something involving a question mark, and I was thinking it'd be nice to relate it to the "reviewed" icon (used by PageTriage and proposed to be used for deferred changes too, see T54510#2875256), so maybe something like the reviewed icon but with a question mark instead of a tick mark?

The first secondary link icon for automatic deferrals (which shows the name of the technology that deferred the edit, either "AbuseFilter" or "ORES") is also a placeholder. We can keep it (it's the fallback icon for secondary links, and we do use it for some of them, like "View log" IIRC), but @Cenarium suggested we could use an icon that represents an automated process and/or an AI. @Pginer-WMF: any ideas for an icon here?

Finally, I worry the wording might be a bit jargon-y / inside-baseball-y. @jmatazzoni, would you like to take a stab at suggesting wording for this?

In T153284#2875499, @Catrope wrote:

The generic "info" icon in the screenshots above is temporary until we come up with something better. @Pginer-WMF: any ideas for an icon for this notification? @Cenarium suggested something involving a question mark, and I was thinking it'd be nice to relate it to the "reviewed" icon (used by PageTriage and proposed to be used for deferred changes too, see T54510#2875256), so maybe something like the reviewed icon but with a question mark instead of a tick mark?

Regarding the icon, I think the central idea is that the change is on hold. I think the concept of time may be more appropriate than the question mark. the later seems to question the quality of the contribution too explicitly before it is evaluated.

Making it part of the same system as the PageTriage one, makes a lot of sense. A proposal (svg at F5086357) is shown below:

I include some versions in context, I considered as part of the exploration:

Time page	Just time	Question page

In T153284#2875499, @Catrope wrote:

The first secondary link icon for automatic deferrals (which shows the name of the technology that deferred the edit, either "AbuseFilter" or "ORES") is also a placeholder. We can keep it (it's the fallback icon for secondary links, and we do use it for some of them, like "View log" IIRC), but @Cenarium suggested we could use an icon that represents an automated process and/or an AI. @Pginer-WMF: any ideas for an icon here?

I agree that an icon can help in this context. Currently it is not clear that "Abuse filter" represents the agent triggering the notification. I recall some discussions in the past to have a "user" icon for bots and other automated entities. I can bring the discussion back with the designers to see if we can have a standard icon. I'm thinking on something along these lines:

Human	Automatic agent

In T153284#2875499, @Catrope wrote:

Finally, I worry the wording might be a bit jargon-y / inside-baseball-y. @jmatazzoni, would you like to take a stab at suggesting wording for this?

Related to this, along the lines of my comment on T54510#2876091, I think we can give a more useful name to the "help" action. Especially thinking on new users that may encounter this as a consequence of their first edit. Having a title such as "Why are edits deferred?" (or even "Why was this edit deferred?" if we can target specific information or the specific section relevant to the current case) could help users to better connect the dots.

i see a number of issues here:

• First, we need to follow our own guidelines and start the process of creating anew notification by filling in the notification form. This will enable us to have an informed conversation. I've pasted the form into the Description, above, and made a start on filling it in.
• Secondly, while a notification is a good idea, it would not seem to be a sufficient means of informing users of what is happening. Deferred changes falls on a) new users and b) anons. The latter don't get notifications at all, and a proportion of the former probably don't know that they're getting them. There should be some type of dialog box or popover, IMHO.
• Finally, for users who do get and read this notification, I think it is not sufficiently informative of what happened. Part of that can be handled by having good links, which we can flesh out in the form. But as this is, it is pretty opaque. What does "deferred for review" mean? What happened to my edit? And in particular, why did this happen? We can't answer all those questions fully in a notification, but I think this message needs to be written more clearly and directly and at least point to the answers.

In T153284#2876195, @Pginer-WMF wrote:

In T153284#2875499, @Catrope wrote:

Finally, I worry the wording might be a bit jargon-y / inside-baseball-y. @jmatazzoni, would you like to take a stab at suggesting wording for this?

Related to this, along the lines of my comment on T54510#2876091, I think we can give a more useful name to the "help" action. Especially thinking on new users that may encounter this as a consequence of their first edit. Having a title such as "Why are edits deferred?" (or even "Why was this edit deferred?" if we can target specific information or the specific section relevant to the current case) could help users to better connect the dots.

If we want to be brief, perhaps the help / question mark icon followed by a label like "Deferred edits" or "Deferred changes"?

In T153284#2876130, @Pginer-WMF wrote:

Regarding the icon, I think the central idea is that the change is on hold. I think the concept of time may be more appropriate than the question mark. the later seems to question the quality of the contribution too explicitly before it is evaluated.

Making it part of the same system as the PageTriage one, makes a lot of sense. A proposal (svg at F5086357) is shown below:

Ooh, I like this icon! Thanks Pau! I completely agree that waiting/clock is a better metaphor than uncertainty/question mark.

I forget what the color conventions for notification icons are (though I know you've written them down somewhere), but the "reviewed" icon in PageTriage is blue. Is it correct/consistent for this one to be grey?

In T153284#2876964, @jmatazzoni wrote:

• Secondly, while a notification is a good idea, it would not seem to be a sufficient means of informing users of what is happening. Deferred changes falls on a) new users and b) anons. The latter don't get notifications at all, and a proportion of the former probably don't know that they're getting them. There should be some type of dialog box or popover, IMHO.

Deferrals are done after the edit is saved for performance reasons (so the time it takes to save an edit isn't impacted), so we can't have a dialog box/popover. Anons not being notified is a limitation of the echo extension, the same applies to reverts, patrols, reviews and so on. I would like very much a way to notify anons, maybe cookie-based temporary notifications, but it should be done in echo. This shouldn't be a blocker, anons aren't notified when reverted, which is stronger than deferred.

In T153284#2877718, @Catrope wrote:

In T153284#2876130, @Pginer-WMF wrote:

Regarding the icon, I think the central idea is that the change is on hold. I think the concept of time may be more appropriate than the question mark. the later seems to question the quality of the contribution too explicitly before it is evaluated.

Making it part of the same system as the PageTriage one, makes a lot of sense. A proposal (svg at F5086357) is shown below:

Ooh, I like this icon! Thanks Pau! I completely agree that waiting/clock is a better metaphor than uncertainty/question mark.

Yes, good idea.

Thanks for filling in the form @Cenarium! Very helpful. @Catrope, do you need more detail about precisely what events trigger the notification, or is that general description sufficient for this purpose?

In T153284#2877718, @Catrope wrote:

I forget what the color conventions for notification icons are (though I know you've written them down somewhere), but the "reviewed" icon in PageTriage is blue. Is it correct/consistent for this one to be grey?

In general it is ok to have diversity of colors for icons in the same family. This makes it easy to distinguish new discussions (in green) from replies on existing topics (in blue) for the discussion family of notifications (using similar speech bubble icons).

The color icon rules don't define very strict rules but they suggest using green for new pieces of information, blue for positive updates on these pieces of information (moving them "forward") and grey for neutral or destructive updates (keeping them as they are or moving it "back" some steps, like an edit being reverted). The case of a deferred change seems to fit on the grey case.

In T153284#2878281, @Cenarium wrote:

Deferrals are done after the edit is saved for performance reasons (so the time it takes to save an edit isn't impacted), so we can't have a dialog box/popover. Anons not being notified is a limitation of the echo extension, the same applies to reverts, patrols, reviews and so on. I would like very much a way to notify anons, maybe cookie-based temporary notifications, but it should be done in echo. This shouldn't be a blocker, anons aren't notified when reverted, which is stronger than deferred.

Regardless of technical limitations, I think it is always good to capture first which is our intent and which would be the ideal user experience in order to see how we can better approach it. Even if we cannot achieve it in all cases, we may improve the support for some users.

If we consider that providing immediate feedback is something desirable, we can consider some approaches to support an early communication. For example:

• We may take advantage of the edit summary step of Visual Editor. We may consider start doing some of the deferral checks as the user fills the edit summary, having the results ready for when the user clicks "save". If the checks take too long, we can use an asynchronous communication mechanism (e.g., notifications) as a fallback for those users that click on the "save" button too fast or those not using Visual Editor.
• We may take advantage of the time some users spend at the article they just edited. After editing, some users check the final result of their edit. If the deferral process completes while the user is still at the article, a more in-context explanation of what happened can be provided, even for anonymous users.

Those are just some possibilities, but my main point is that if we don't capture the intent behind a specific interaction pattern we may limit our possibilities.

Deferrals can occur from a variety of sources and at any time after the edit is saved. Bots can defer via the API the same way they can rollback for example. And ORES uses a job queue to score edits. Only AbuseFilter could perform a deferral at the time the edit is saved, but again this would slightly delay saving, and there's no way to do this before the actual save attempt (it's not possible at the edit summary step of VE or preview). If immediate feedback is needed on an edit, there's already the possibility of using the warn action of AbuseFilter.

In T153284#2876159, @Pginer-WMF wrote:

I agree that an icon can help in this context. Currently it is not clear that "Abuse filter" represents the agent triggering the notification. I recall some discussions in the past to have a "user" icon for bots and other automated entities. I can bring the discussion back with the designers to see if we can have a standard icon. I'm thinking on something along these lines:

Human	Automatic agent

This looks good too, thanks. would you have a svg version of it?

In T153284#2880207, @Pginer-WMF wrote:

In general it is ok to have diversity of colors for icons in the same family. This makes it easy to distinguish new discussions (in green) from replies on existing topics (in blue) for the discussion family of notifications (using similar speech bubble icons).

The color icon rules don't define very strict rules but they suggest using green for new pieces of information, blue for positive updates on these pieces of information (moving them "forward") and grey for neutral or destructive updates (keeping them as they are or moving it "back" some steps, like an edit being reverted). The case of a deferred change seems to fit on the grey case.

Right, I mean consistency with the rules, I didn't mean to imply that the color difference between "reviewed" and "deferred" was inconsistent. If blue means positive update and grey means neutral/destructive, then it makes total sense for "reviewed" to be blue and "deferred" to be grey.

In T153284#2880249, @Pginer-WMF wrote:

If we consider that providing immediate feedback is something desirable, we can consider some approaches to support an early communication. For example:

• We may take advantage of the edit summary step of Visual Editor. We may consider start doing some of the deferral checks as the user fills the edit summary, having the results ready for when the user clicks "save". If the checks take too long, we can use an asynchronous communication mechanism (e.g., notifications) as a fallback for those users that click on the "save" button too fast or those not using Visual Editor.

Showing AbuseFilter results in the VE save dialog (or the wikitext editor preview screen!) would require having a dry-run mode in AbuseFilter, and I don't AF supports that currently. I'm also not sure if that's something that would be desirable, though perhaps the dry-run mode could be limited to predicting whether certain actions will be taken (e.g. preventing the edit, deferring it, or allowing it after warning the user) while not predicting others (e.g. tagging). This could also improve VE's save performance for edits that AF shows a warning for, as those have to be submitted twice.

• We may take advantage of the time some users spend at the article they just edited. After editing, some users check the final result of their edit. If the deferral process completes while the user is still at the article, a more in-context explanation of what happened can be provided, even for anonymous users.

Those are just some possibilities, but my main point is that if we don't capture the intent behind a specific interaction pattern we may limit our possibilities.

It would be nice to figure out a way to present information from deferred (in the DeferredUpdate sense, not the "deferred edit" sense) post-save processing to the user. That would let us tell them about a number of things, not only deferrals and AbuseFilter actions, but also mentions (see T68078#2584419); and I'm sure there are other things.

In T153284#2878452, @jmatazzoni wrote:

Thanks for filling in the form @Cenarium! Very helpful. @Catrope, do you need more detail about precisely what events trigger the notification, or is that general description sufficient for this purpose?

I don't need a description of when the notification is triggered, because I don't need to write the code, because @Cenarium has already written it :)

@jmatazzoni: Thanks for suggesting different text. I'm happy with the header message, I think adding the word "publication" makes the meaning of deferral a lot clearer. As for the body message, I'm generally happy with that too, but I wonder if "screening software" is an acceptable generalization for the sources of deferral. @Cenarium: what are your thoughts on that, and Joe's proposed language more generally?

but I wonder if "screening software" is an acceptable generalization for the sources of deferral

Are you saying you want to be more specific? As in "Abuse filter found a possible problem"? I don't see that as helpful, and could actually give clues to vandals about how to avoid. Or are you saying that you don't think that Cluebot, ORES and Abuse Filter are "screening software." While that might be true in the case of ORES, it seems to me that it is acting as such in this case, so it works for me. As for the other two, screening seems to be just what they're doing. But I suppose we could call it "Quality evaluation software" or "Automated quality testing" or "Our robots." In fact, that last one puts a friendly spin on this message that I rather like. How about something along these lines:

Publication of your edit on Pagename was deferred, pending review.
Our robots found a possible problem, so a human will have a look.

OMG, I just noticed the "automatic agent" icon. It would definitely reduce the sting of deferral if users got that with the messaging just suggested:

In T153284#2881842, @jmatazzoni wrote:

but I wonder if "screening software" is an acceptable generalization for the sources of deferral

Are you saying you want to be more specific? As in "Abuse filter found a possible problem"?

Sorry, I should have been clearer. I meant that "screening software" might be too specific, and I was wondering if there was a case (or there could reasonably be one in the future) where a deferral did not come from "screening software". However, deferral as currently proposed is limited to accounts with the bot flag and automated tools (AbuseFilter and ORES), so I suppose it'd be OK to assume that it came either from an automated process (either an on-wiki one, or on interacting with the wiki via a bot-flagged user account).

In T153284#2881549, @Cenarium wrote:

This looks good too, thanks. would you have a svg version of it?

Yes. Here it is the SVG version:

I shared the ticket with the designers for input, but if there are proposed changes we can update it later.

Change 316957 had a related patch set uploaded (by Cenarium):
Add echo notification when deferred

https://gerrit.wikimedia.org/r/316957

In T153284#2885802, @Pginer-WMF wrote:

Yes. Here it is the SVG version:

robot.svg1 KBDownload

Also the png version in black (as the other action icons use): F5098846

I shared the ticket with the designers for input, but if there are proposed changes we can update it later.

Thanks. I added the deferred changes icon after resizing it to 24x24. However, I couldn't get the robot icon to display in echo, even after resizing it to 24x24.

FYI, I will go on vacation soon and won't be back until mid January.

So, @Pginer-WMF, should I change the spec above to show that the icon is "automatic agent"?

And, partly in light of that change, I've decided I like the body text I suggested above, which has a friendly sound I think. The full message would look like this:

Publication of your edit on Pagename was deferred, pending review.
Our robots found a possible problem, so a human will have a look.

When I say this sounds friendlier, I think what I mean is that a) reference to "our robots" has a lighthearted sound that b) helps reassure people we're not giving too much credence to this automatic process. Does anyone object to me updating the body text with that? With this message and that cute robot face, how could anyone get mad at us?

In T153284#2888491, @jmatazzoni wrote:

So, @Pginer-WMF, should I change the spec above to show that the icon is "automatic agent"?

Sounds good. Also, the proposed text works for me. It uses the concept of publishing and review to clarify what (didn't) happen and what's happening next. The tone in the use of robots seems also a good approach to avoid the consequences of accidentally blaming well-intentioned users.

Icon and body text updated in Description form, above.

See my comment in T58828#2902925. I plan on providing support in Echo for anon notifications, which we'll then be able to use here.