I guess step one is to get the code into Git/Gerrit. CC'ing Sam and Chad about that.

(In reply to comment #1)

I guess step one is to get the code into Git/Gerrit. CC'ing Sam and Chad about
that.

On it, will be done shortly.

(In reply to comment #2)

(In reply to comment #1)

I guess step one is to get the code into Git/Gerrit. CC'ing Sam and Chad about
that.

On it, will be done shortly.

Done, https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/Drafts

Consensus?

(In reply to comment #4)

Consensus?

I believe this extension falls under general MediaWiki enhancements and doesn't require local community consensus to enable.

Do you feel otherwise? If so, how would you recommend moving forward?

(In reply to comment #5)

(In reply to comment #4)

Consensus?

I believe this extension falls under general MediaWiki enhancements and doesn't
require local community consensus to enable.

Do you feel otherwise? If so, how would you recommend moving forward?

I'd agree with you. It should be an unobtrusive benefit for anyone with a browser that works with it.

And even then, it's got a preference to turn it on/off

-shell; nothing for ops to do yet. Extension needs review and improvements first.

sumanah wrote:

Trevor is looking at this extension for review & improvement. See https://gerrit.wikimedia.org/r/#/q/project:mediawiki/extensions/Drafts -owner:L10n-bot,n,z .

swalling wrote:

(In reply to comment #4)

Consensus?

This is also a feature that someday would have been tested anyway for new page creation. See also: https://www.mediawiki.org/wiki/Article_Creation_Workflow/Design

sumanah wrote:

Trevor is continuing to work on Drafts for about one day a week, fixing bugs, improving database handling, adding ResourceLoader and jQuery integration, and so on.

Could we see this put up on a labs instance somewhere? I have several concerns, but I'm not sure how valid they are because I can't actually see an example of the software in use - which is an issue.

TL;DR, I really oppose turning this on with the way it's formatted now and using the process that's being discussed.

*For more detail on each point; this is not, as MzMcBride claims, "a general MediaWiki enhancement" unless you want to expand the definition to cover pretty much anything. It introduces a completely new workflow, alters existing ones and sticks a big button on the (already overfilled) page that we know all editors are going to see. There is no way this can be turned on without a community discussion unless you want a raging storm of anger hurled in the direction of whoever hits the big red button.

*The UI elements clash with current thinking about the direction that we're going in. The Micro Designs Improvement project is currently working on the edit window as we speak, and plans to do a couple more iterations given the opportunity. I'd rather not throw two competing philosophies of design into the mix - that works if they're from the same team, but I worry we'd end up with (at best) an inconsistent UI and (at worst) an active clash.

*This really doesn't seem an efficient way to do things. What's the use case here, exactly? If it's "people would like to save a draft in case they lose their work", save the draft automatically after [number] of minutes or seconds rather than requiring them to actually make a decision, and then just void any drafts after [other number] of minutes or seconds. If it's "we want to sort how confusing the existing setup is by offering functions found on other sites", integrate that into the existing workflows to avoid button bloat. At https://www.mediawiki.org/wiki/Extension:Drafts I'm seeing edge cases discussed and the workflow discussed...but not how this in any way can be integrated into how Wikipedia currently works, or how we'd like it to work, or what exactly the use case is for this software. If there is a use case, it needs to be communicated. If there isn't, we shouldn't be turning it on.

*Can someone point me to, if not a use case, the user demand for this feature? One power user requesting it on bugzilla is not a case for change.

This is not to say that it isn't useful, or that this type of feature isn't the sort of thing we should be looking at - as mentioned above, reforming the editing workflows is something that is being discussed and worked on. I'm just skeptical that this particular information, particularly deployed in this fashion, is going to benefit more than it costs.

I've asked on Portuguese Wikipedia a few days ago:
http://pt.wikipedia.org/wiki/Project:Esplanada/propostas/Instalar_extensão_para_salvar_rascunhos_(1set2012)?uselang=en

So far, no one opposed it, but one of the users asked if we could test it somewhere. According to
http://www.gossamer-threads.com/lists/wiki/wikitech/278043
the extension was installed by [[User:Petrb]] on
http://en.wikipedia.beta.wmflabs.org/wiki/Special:Drafts
but the link is not working right now.

(In reply to comment #12)

*Can someone point me to, if not a use case, the user demand for this feature?

The use case and user demand is the hundreds or thousands of users saving their edits every couple of minutes and so for fear of losing everything.
I'm not a developer but the points you raise, although very important, don't seem to be blockers, rather marginal ones.
For instance, removing the button by default seems a reasonable solution, as long as the user understands what's going on (just a small notice every time the draft is saved?). A configuration variable could be used to enable the button for wikis which desire it.

Really? Who makes these saves? And if this is a common use case, could we have an on-wiki demonstration that people need or want it? :)

I'm not a developer either. However, I consider needlessly complicating the UI interface or introducing new workflows without consulting people in advance or having a firm use case that we actually know occurs to be a core problem with what's being discussed - regardless of what's being discussed.

(In reply to comment #15)

Really? Who makes these saves? And if this is a common use case, could we have
an on-wiki demonstration that people need or want it? :)

A DB query will give you plenty of examples. ;)

I'm not a developer either. However, I consider needlessly complicating the UI
interface or introducing new workflows without consulting people in advance or
having a firm use case that we actually know occurs to be a core problem with
what's being discussed - regardless of what's being discussed.

As I already said, I don't think this introduces new workflows, except when the workflow is "rewrite everything from scratch because you've lost all you wrote" or "save every 30 s to avoid losing everything". And no, I can't imagine anyone who could dislike this feature intervening in such cases.

I have to say, I'm with Oliver here. I have concerns about deployment on a wide scale without some further feature development and (more importantly) community input and buy-in. It seems to me that there are policy and governance changes that also need to be introduced at the same time, and it would be courteous to let the community work on those with some warning.

I also have UI challenges with this. It's not immediately transparent how it works to a new user (where is the integration to contribution history or recent changes?), and (while I haven't seen the documentation) I suspect that the documentation is also not user-ready. I would hope that the UI takes advantage of some of the knowledge gained through the past year's experimentation rigor, rather than charting a course that may or may not make sense.

With all that said, I agree with MZMcBride's intent in moving this forward - limbo is a terrible place, and when promising tools are stuck there, it's frustrating. We SHOULD improve the pipeline; but let's do it intelligently.

Philippe

(In reply to comment #15)

Really? Who makes these saves? And if this is a common use case, could we have
an on-wiki demonstration that people need or want it? :)

Ok. I just created a topic on enwiki for this:
https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Allowing_users_to_save_drafts_of_their_work

(and there was the other one on ptwiki, mentioned on comment 13).

(In reply to comment #12)

TL;DR, I really oppose turning this on with the way it's formatted now and
using the process that's being discussed.

Most of your comment reads as "I'm bored and looking for a fight" with a hint of "why wasn't I consulted?" If you want to start petty disputes on Bugzilla with me, you really ought to do better research into the battles you pick. This isn't my brainchild or my code. It's a missing MediaWiki feature that would be particularly helpful to have on Wikimedia wikis.

*For more detail on each point; this is not, as MzMcBride claims, "a general
MediaWiki enhancement" unless you want to expand the definition to cover pretty
much anything.

It seems like a fairly standard enhancement to me.

It introduces a completely new workflow, alters existing ones
and sticks a big button on the (already overfilled) page that we know all
editors are going to see. There is no way this can be turned on without a
community discussion unless you want a raging storm of anger hurled in the
direction of whoever hits the big red button.

I'm not sure I understand what you're saying here. Microsoft Word has auto-save; WordPress has auto-save; modern Web browsers have auto-save. Who do you think will be shouting about this being enabled? Is there any evidence of this being a controversial feature?

*The UI elements clash with current thinking about the direction that we're
going in. The Micro Designs Improvement project is currently working on the
edit window as we speak, and plans to do a couple more iterations given the
opportunity. I'd rather not throw two competing philosophies of design into the
mix - that works if they're from the same team, but I worry we'd end up with
(at best) an inconsistent UI and (at worst) an active clash.

I'm not sure how this is relevant. Other parts of the edit view need work, yes.

*This really doesn't seem an efficient way to do things. What's the use case
here, exactly? If it's "people would like to save a draft in case they lose
their work", save the draft automatically after [number] of minutes or seconds
rather than requiring them to actually make a decision, and then just void any
drafts after [other number] of minutes or seconds.

This extension pretty much does exactly that, doesn't it? I don't think the current version of this extension is as heavy-handed about auto-discarding drafts as you suggest it could be, but that seems like a feature to me, not a bug.

If it's "we want to sort how
confusing the existing setup is by offering functions found on other sites",
integrate that into the existing workflows to avoid button bloat. At
https://www.mediawiki.org/wiki/Extension:Drafts I'm seeing edge cases discussed
and the workflow discussed...but not how this in any way can be integrated into
how Wikipedia currently works, or how we'd like it to work, or what exactly the
use case is for this software. If there is a use case, it needs to be
communicated. If there isn't, we shouldn't be turning it on.

I didn't follow most of this section, so I'm not quite sure how to respond.

*Can someone point me to, if not a use case, the user demand for this feature?
One power user requesting it on bugzilla is not a case for change.

You can't formulate a use-case for a drafts feature in your head? Interesting. You've been editing Wikipedia for how many years now? You've never wanted to create a draft of something? :-)

As for the "one power user" comment, when you look at the history of the development of the extension (which you apparently didn't) and you look at related bugs (such as bug 19909), you can see that your comment doesn't make much sense. I'm not sure what you gain by trying to act as though there's only person pushing for auto-save functionality on Wikimedia wikis.

This is not to say that it isn't useful, or that this type of feature isn't the
sort of thing we should be looking at - as mentioned above, reforming the
editing workflows is something that is being discussed and worked on. I'm just
skeptical that this particular information, particularly deployed in this
fashion, is going to benefit more than it costs.

"The extension needs a bit of work before deployment. Water is also wet. You can safely ignore the last few paragraphs I wrote."

(In reply to comment #17)

I have to say, I'm with Oliver here. I have concerns about deployment on a
wide scale without some further feature development and (more importantly)
community input and buy-in. It seems to me that there are policy and
governance changes that also need to be introduced at the same time, and it
would be courteous to let the community work on those with some warning.

Can you elaborate on the policy and governance changes you envision?

I also have UI challenges with this. It's not immediately transparent how it
works to a new user (where is the integration to contribution history or recent
changes?), and (while I haven't seen the documentation) I suspect that the
documentation is also not user-ready.

They're private (per-user) drafts. I'm not sure if this was apparent when you wrote this comment.

It'd be nice if this extension could be enabled on https://test.wikipedia.org or similar. A lack of a working demonstration is hurting the conversation, I fear.

(In reply to comment #20)

It'd be nice if this extension could be enabled on https://test.wikipedia.org
or similar. A lack of a working demonstration is hurting the conversation, I
fear.

Okeyes said at
[[WP:Village_pump_(proposals)#Allowing_users_to_save_drafts_of_their_work]]
that (the version 0.1.1) of the extension is enabled on this wiki:
http://kubo.wmflabs.org/wiki/Main_Page
I don't know if this is the most up to date version.

As a user, this seems useful.

What to do about edit conflicts might be an issue, but even a short warning not to leave it too long (and why) upon saving a draft could be enough for that, and if a page is too highly edited, maybe just removing the draft option entirely on those would be in order? Could confuse people a little at first, but they'd probably catch on eventually since a draft on, say, enwp's WP:ANI would make NO BLOODY SENSE in the first place.

Just suddenly turning on automatic drafts could also pose problems due to current habits of starting edits and then thinking better of them and leaving, as opposed to using the cancel button, but editors can be retrained. On the other hand, since editing and then previewing is not only the only way to test things without saving them, but also a really convenient one, that might actually be an issue... but if all else fails, a user option to turn the automatic drafts off should fix that pretty effectively.

And the only part of this that seems like it might go out of general enhancement territory and into OUTRAGE WHY YOU CHANGE THIS?! territory is the automatic drafts. But while a decision on the parts of the projects themselves whether or not enable that part at all might make them a little happier, why should be necessary, especially if users have the option to just turn it off themselves if they don't like it?

(In reply to comment #8)

Trevor is looking at this extension for review & improvement. See
https://gerrit.wikimedia.org/r/#/q/project:mediawiki/extensions/Drafts -owner:L10n-bot,n,z
.

Couple of questions:

1. Will these be indexed?
2. Can they be deleted?
3. Do they show up in Recent Changes?
4. Is suppression enabled?
5. How long between creation and "automatic expiry"? Is this configurable by user or only within the code?
6. Will the edits made in a "draft" count as edits statistically?
7. When drafts are "saved" into the page, is the edit history of the draft incorporated into the page history? If so, is there a way to indicate that it was made on a "different" page and not directly into the article? [Note that one of the biggest issues in merging page histories is that it is pretty much impossible to correctly identify what past versions of the page looked like.]
8. When drafts expire, is that information completely inaccessible, or accessible only under specific circumstances? [It's important to be able to re-access any material that was *ever* publicly viewable because we never know which bits will wind up requiring some sort of response in the future.]
9. Will drafts be included in dumps?
10. What evidence is there that *users* want this feature?

I think there is a faulty assumption behind some of the discussion here, that users are saving frequently because of fear of losing their work. This isn't true; users save frequently because (a) they want to see their work in the context of the page being edited, (b) there is a systemic problem with saving work because of time-outs or the system logging them out/otherwise not accepting edits, that will not be rectified by this and (c) they are beefing up their edit counts without using scripts.

The drafts saved by a user are not publicly visible and as such do not show up on
recent changes. The author can delete them at any time by clicking on "Discard" either while editing a page for which he saved a draft or while accessing all his personal drafts by Special:Drafts.

(In reply to comment #24)

The drafts saved by a user are not publicly visible and as such do not show up
on
recent changes. The author can delete them at any time by clicking on "Discard"
either while editing a page for which he saved a draft or while accessing all
his personal drafts by Special:Drafts.

Thanks, mybugs, for answering #2 and #3. So the response to #1 should be no because the draft is not publicly visible, and #4 & #8 would be moot if it is seen only by the user.

So this leaves:

5. How long between creation and "automatic expiry"? Is this configurable by

user or only within the code?

6. Will the edits made in a "draft" count as edits statistically?
7. When drafts are "saved" into the page, is the edit history of the draft

incorporated into the page history? If so, is there a way to indicate that it
was made on a "different" page and not directly into the article? [Note that
one of the biggest issues in merging page histories is that it is pretty much
impossible to correctly identify what past versions of the page looked like.]

9. Will drafts be included in dumps?
10. What evidence is there that *users* want this feature?

I learned long ago not to assume too much here.

For the record, I'd strongly urge that the first response to any Bugzilla that will result in a change to the UI should be "Please point me to the community discussion that recommends this change." I'm not sure why the WMF would be investing 20% of an engineer's time without any evidence that [multiple] communities are interested in activating an extension. There are plenty of extensions around that, simply put, aren't used because they're unwanted or have a negative impact on a project. Unexpected and undiscussed UI changes have repeatedly caused significant discord between communities and Engineering.

sumanah wrote:

(In reply to comment #25 by Risker)

For the record, I'd strongly urge that the first response to any Bugzilla that
will result in a change to the UI should be "Please point me to the community
discussion that recommends this change." I'm not sure why the WMF would be
investing 20% of an engineer's time without any evidence that [multiple]
communities are interested in activating an extension. There are plenty of
extensions around that, simply put, aren't used because they're unwanted or
have a negative impact on a project. Unexpected and undiscussed UI changes
have repeatedly caused significant discord between communities and Engineering.

Thanks for the comment, Risker, and thanks to everyone in this thread for all your comments.

I'm sorry; this problem is at least partially my fault. Just before my vacation this summer, I asked for some help maintaining the review queue (for new extensions to potentially be deployed on WMF sites: https://www.mediawiki.org/wiki/Review_queue ). The Drafts extension got added to this list, and in late August I accidentally went straight into requesting code review/improvements from Trevor, using his "20%" time ( https://www.mediawiki.org/wiki/Wikimedia_engineering_20%_policy ) . Per our procedures at https://www.mediawiki.org/wiki/Writing_an_extension_for_deployment#Design_review , I should have instead gotten Drafts a design review first, from our product management or design team. As the "design review" procedure says, "This will start a conversation about the design of your feature, which should include community consensus on the wiki where the extension would be deployed."

The design team is currently discussing Drafts on the design mailing list:

http://lists.wikimedia.org/pipermail/design/2012-September/thread.html#130

and I will update that list, Trevor, and the review queue to ask for that design review.

Thanks, and my apologies again.

(In reply to comment #26)

I'm sorry; this problem is at least partially my fault. [...]

And just to clarify, I take that portion of fault for myself because I'm the one who added the extension to the queue page, perhaps misleading someone. I did so because I well remember this coming from real usability issues found at the time, being tested and receiving unanimously positive comments which can in part still be found at https://usability.wikimedia.org (I agree that the lack of a live prototype is confusing those who can't remember the 2009 one). I think the documentation already explains everything that was requested, but maybe I'm reading too much into it.
Anyway, as Sumana notes, design is now being re-reviewed by the wider design team.

stillwaterising wrote:

(In reply to comment #25)

For the record, I'd strongly urge that the first response to any Bugzilla that
will result in a change to the UI should be "Please point me to the community
discussion that recommends this change." I'm not sure why the WMF would be
investing 20% of an engineer's time without any evidence that [multiple]
communities are interested in activating an extension. There are plenty of
extensions around that, simply put, aren't used because they're unwanted or
have a negative impact on a project. Unexpected and undiscussed UI changes
have repeatedly caused significant discord between communities and Engineering.

The percentage of editors reading message boards like Village pump (idea lab) has to be a very small fraction. The ratio of developers to normal users reading and responding will be disproportionate. Village Pump does not represent what the average user wants.

Successful companies looking to invest in new services or products will perform studies, focus groups, and surveys. What I propose is a survey consisting of 250 randomly-selected recently active auto-confirmed users.

Users could be shown images or videos as well having written descriptions of proposed changes to the UI and ask for how interested, how useful, how effective, etc. they think it will be for them.

Smaller focus groups can be organized where users are asked to go to a test wiki and perform certain tasks and answer questions about their experience.

I agree that this is a useful extension, but this goes beyond code review. The extension is not deployable in its current state and needs significant work. I've added some dependencies.

I don't think this is doable as a 20% time project for Trevor, so unless there are more folks who want to chip in improvements, I'd put this on hold for now from a WMF standpoint.

(In reply to comment #29)

I agree that this is a useful extension, but this goes beyond code review. The
extension is not deployable in its current state and needs significant work.
I've added some dependencies.

Yes, completely agreed. I hadn't realized this reality when I filed this bug.

I actually considered marking this bug as "resolved later" over the weekend, but now that it has some dependencies and lacks the "shell" keyword, I think leaving this bug open as a tracking bug should work well. Thanks for looking into this, Erik!

capmo700 wrote:

(In reply to comment #19)

... modern Web browsers have auto-save.

That's a good point. A couple of years ago this extension would have been really appreciated by me, but nowadays it has become almost superfluous, thanks to the improvement of the browsers' auto-save capabilities (Firefox, for one, can recover what you have typed even in the case of a power failure).

Another thing to consider is that, after such feature is deployed, users will generally feel safer about starting an edit and postponing the save for hours later, posing a higher risk of edit conflicts (especially in larger wikis as Wikipedia).

(In reply to comment #31)

(In reply to comment #19)

... modern Web browsers have auto-save.

That's a good point. A couple of years ago this extension would have been
really appreciated by me, but nowadays it has become almost superfluous, thanks
to the improvement of the browsers' auto-save capabilities (Firefox, for one,
can recover what you have typed even in the case of a power failure).

I'm not aware of other browsers having the same capabilities, which I anyway still see myself failing, not seldom. There are also many other chances to lose an edit with actions that the browser can consider intentional. In short, it's not a valid assumption that the wiki editor is using a human-like smart browser.

Another thing to consider is that, after such feature is deployed, users will
generally feel safer about starting an edit and postponing the save for hours
later, posing a higher risk of edit conflicts (especially in larger wikis as
Wikipedia).

I doubt so, and anyway it's easily configurable as far as I can see.

I agree that browser auto-save isn't a sufficient alternative to this feature. We still need this, and help whipping the extension into shape is very much appreciated.

armbrust wrote:

There was a discussion on the Proposal Village Pump on the English Wikipedia (http://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)/Archive_94#Allowing_users_to_keep_private_drafts_of_their_work), where this feature was cleary supported.

....by six people ;p.

Visual editor brings new dataloss scenarios due to its beta status. If VE could autosave the article as a draft, and could open the draft in VE, this would allow switching editor.

The demo of this extension (http://kubo.wmflabs.org/wiki/Main_Page) now returns 404. There are some non-WMF wikis using this extension: http://wikiapiary.com/wiki/Extension:Drafts

Is it enabled on a WMF test wiki with VE? Could it be enabled on mediawiki.org in order to encourage dev's playing with it?

From a quick look at the code, a prototype of VE-support of Drafts can be done easily with only user: space custom JavaScript.

Proper VE-support would need a new API call to minimise data being sent back and forth.

(In reply to comment #37)

Is it enabled on a WMF test wiki with VE? Could it be enabled on
mediawiki.org in order to encourage devs playing with it?

Greg: do you know?

Depends by what you mean as a wmf test wiki. Its not on the cluster, as its not branched. I don't think it is on beta either, but you van confirm it yourselves via NOC or mediawiki-config

Shouldn't be deployed on cluster until code has had relevant code review actioned against it, and any necessary remedial action completed.

Well, and until Features and Product have confirmed it's something they're interested in supporting (unless Platform devs are comfortable explicitly saying 'yes, we will make sure it works forevermore')

(In reply to comment #39)

Shouldn't be deployed on cluster until code has had relevant code review
actioned against it, and any necessary remedial action completed.

Hmm, right, I forgot that this bug has a bunch of outstanding dependencies.

John V.: we'll need to find someone to work on the extension in order to eventually get it deployed on any Wikimedia wikis.

(In reply to comment #37)

Is it enabled on a WMF test wiki with VE? Could it be enabled on
mediawiki.org
in order to encourage dev's playing with it?

So I set up an instance at http://drafts.instance-proxy.wmflabs.org/wiki/Main_Page and attempted to install VE, except setting up parsoid failed with some cryptic error I didn't spent time debugging.

Digging through the code, there's a lot of cleanup (deprecated stuff, (ab)use of globals, etc) that needs to be done and a start has been made at https://gerrit.wikimedia.org/r/#/c/29749/

It would also be helpful if there are people willing to do code review, it seems that's whats stalled the development of this extension in the past.

(In reply to comment #42)

It would also be helpful if there are people willing to do code review, it
seems that's whats stalled the development of this extension in the past.

It seems Yuvi is willing. You can also e-mail wikitech-l if you can't find anyone. :-)

Just a quick note, in case people are pushing hard on getting this done so it can be used for VisualEditor, it's not the direction we're looking at taking for VE.

• We would need something that is client-side, not server-side (this makes things faster and avoids the really icky legal issues).
• We would need something that lets us persist VE linear-models, or at least Parsoid HTML RDFa (otherwise each save will take dozens of seconds of computation, slowing clients down and adding to burdens on the cluster; until server-side storage of Parsoid HTML RDFa is undertaken, there's nowhere really appropriate for this to be saved for the Drafts extension to implement, which is currently wikitext-based)
• We would want (but not strictly /need/) something very flexible, so we could persist undo stacks, etc. which could add a huge amount of complexity to the Drafts extension for just one use case/client.

Though I think that the Drafts extension could be useful for wikitext users, I don't think it's right for VisualEditor; see bug 55370 for VisualEditor's take.

I'm not sure this is should be discussed on this bug but...
(In reply to comment #44)

Just a quick note, in case people are pushing hard on getting this done so it
can be used for VisualEditor, it's not the direction we're looking at taking
for VE.

Thanks for the comment. Really I'm just trying to get this extension to not suck ;-)

• We would need something that lets us persist VE linear-models, or at least

Parsoid HTML RDFa (otherwise each save will take dozens of seconds of
computation, slowing clients down and adding to burdens on the cluster; until
server-side storage of Parsoid HTML RDFa is undertaken, there's nowhere
really
appropriate for this to be saved for the Drafts extension to implement, which
is currently wikitext-based)

Technically Drafts is set up to store anything that is text based, and I think HTML RDFa can be serialized into text (or is already text)? We might need to add a column for contentmodel in the drafts db schema though...

• We would want (but not strictly /need/) something very flexible, so we

could
persist undo stacks, etc. which could add a huge amount of complexity to the
Drafts extension for just one use case/client.

Ideally the Drafts extension would also have some kind of history/undo, but this is kind of a slippery slope and at some point you end up with essentially private user sandboxes.

Though I think that the Drafts extension could be useful for wikitext users,
I
don't think it's right for VisualEditor; see bug 55370 for VisualEditor's
take.

(In reply to comment #44)

Just a quick note, in case people are pushing hard on getting this done so it
can be used for VisualEditor, it's not the direction we're looking at taking
for VE.

• We would need something that is client-side, not server-side (this makes

things faster and avoids the really icky legal issues).

Are these 'icky legal issues' explained somewhere? Are you referring to the need for explicit copyright signoff before *distribution*? Why cant that be delayed until the real save button is pressed?

A client-side solution is not going to go down well. People switch between computers and browsers, use internet cafes, etc. Wikipedia and VE are a web-app; ordinary users will expect auto-save to persist across their user-agents. A client side solution isnt auto-save; it is a data-recovery mechanism only, which is still useful in its own right, but each client will have its own quirks, etc.

• We would need something that lets us persist VE linear-models, or at least

Parsoid HTML RDFa (otherwise each save will take dozens of seconds of
computation, slowing clients down and adding to burdens on the cluster; until
server-side storage of Parsoid HTML RDFa is undertaken, there's nowhere
really
appropriate for this to be saved for the Drafts extension to implement, which
is currently wikitext-based)

There is no client-side hit required to persist VE as wikitext. VE already has an API call that accepts HTML RDFa and produces wikitext; VE can add an API call that accepts HTML RDFa and persists it as wikitext into the Drafts tables.

Not sure that there is going to be an overall additional burden to the servers either. Converting HTML RDFa to wikitext will be a memory&processor hit to the app servers, whereas persisting HTML RDFa into the database is an IO hit as compared to wikitext.

Besides, what percentage of overall server load is spent on processing page saves to content pages (excluding rendering as autosave in VE doesnt need a render hit)? I'm guessing that isn't where the WMF resources are being pinched.

• We would want (but not strictly /need/) something very flexible, so we

could
persist undo stacks, etc. which could add a huge amount of complexity to the
Drafts extension for just one use case/client.

I agree with Legoktm; it would be great if the source editor also had some features added to it. undo stacks is a major part of the problems holding up the deployment of Drafts; Drafts needs to handle multiple revisions of the same page being saved as a draft. I hear the majority of the editor base tend to use the source editor most of the time ;-)

[Please note that I am not a lawyer and not currently working in a criminal prosecution or national security context; this is not legal advice, just based on my professional experience.]

(In reply to comment #46)

(In reply to comment #44)

Just a quick note, in case people are pushing hard on getting this done so it
can be used for VisualEditor, it's not the direction we're looking at taking
for VE.

• We would need something that is client-side, not server-side (this makes

things faster and avoids the really icky legal issues).

Are these 'icky legal issues' explained somewhere? Are you referring to the
need for explicit copyright signoff before *distribution*? Why cant that be
delayed until the real save button is pressed?

Sorry - when I quickly wrote that yesterday, I assumed they were obvious, but that probably says more about me and my priors than anything else (I have a background in criminal prosecution and central government/security) - my apologies. :-)

If we create a server-side private drafts facility, we invite malefactors to (ab)use the facility to store and (using accounts with a shared password) distribute:

• Terrorism-related materials;
• Child pornography (even if the Drafts are 'just' text, e.g. base64 encoding of images);
• Foreign intelligence service / espionage channels (let's please not make the Wikileaks comparisons true);
• Other criminal information; and
• Civil illicit information (e.g. breaking NDAs, corporate espionage, …)

All this kind of stuff is potentially uploaded to the public fora of Wikimedia's wikis (talk pages, etc.), but (because people are aware that it's public) it is much less frequently put there, and so is at a level where it can generally be dealt with by the existing community processes. A private cache of drafts that isn't visible to any other user means that the community couldn't police it, or flag for suppression/other admin action.

Because of this, *even if no terrorist/pædophile/spy/criminal/mole ever did this*, WMF would get many, many subpœnæ, warrants, clandestine requests (National Security Letters, etc.) and other legal instruments which would have the effect of hugely increasing the cost to comply with these matters. I appreciate that lots of the other top-10 websites have server-side drafts facilities, but they have lots more money to throw away like this, and (allegedly) direct NSA/FBI/… access to their servers anyway, reducing the legal compliance cost (but something to which we would never agree, of course). People running less-visible services run into this problem from time to time, and in some cases end up shutting down their services; we can't take the risk that we'd need to do this at Wikimedia.

A client-side solution is not going to go down well. People switch between
computers and browsers, use internet cafes, etc. Wikipedia and VE are a
web-app; ordinary users will expect auto-save to persist across their
user-agents. A client side solution isnt auto-save; it is a data-recovery
mechanism only, which is still useful in its own right, but each client will
have its own quirks, etc.

I appreciate that this is a pain, but the drafts would probably need to only last for a few hours anyway (it's intentionally a DR solution, as you say) - Wikimedia is in the business of providing wikis, and long-term drafting is completely anti-thetical to the concept of a wiki, where "the page is the draft". If you have a change which is an improvement, even if it's not all it could possibly be, save it so others can collaborate with you. I would be very troubled if we were to move in a direction of encouraging such anti-wiki behaviours.

• We would need something that lets us persist VE linear-models, or at least

Parsoid HTML RDFa (otherwise each save will take dozens of seconds of
computation, slowing clients down and adding to burdens on the cluster; until
server-side storage of Parsoid HTML RDFa is undertaken, there's nowhere
really appropriate for this to be saved for the Drafts extension to implement,
which is currently wikitext-based)

There is no client-side hit required to persist VE as wikitext. VE already
has an API call that accepts HTML RDFa and produces wikitext; VE can add an API
call that accepts HTML RDFa and persists it as wikitext into the Drafts
tables.

I'm sorry, this is not correct. The API call in VE that you mention is a server-side request. The process to save (or get a wikitext diff) works roughly as follows:

• VE client converts data model's linear model into Parsoid HTML RDFa (this can take a noticeable amount of processing time for complex pages)
• VE client establishes a connexion with the (very thin) server-side VisualEditor code, and transmits the HTML RDFa to it (noticeable network time for long pages)
• VE server establishes a connexion with the Parsoid service, and transmits the HTML RDFa to it (server-side IO load)
• Parsoid pulls from storage or the MW API the previous version of the page and other data so that it can as-cleanly-as-possible serialise (server-side IO load)
• Parsoid serialises the HTML RDFa from the client into wikitext based on this information (server-side CPU load)
• Parsoid responds down the connexion with VE server and transmits the wikitext to it (server-side IO load)
• VE server establishes a connexion with the MW cluster, and transmits the wikitext to it (server-side IO load)
• MW saves the wikitext, or fails and returns an error code with why (server-side CPU load)

This whole process can take multiple seconds (as you will see every time you press "save" in VE), and the first step is a significant CPU drain locally; I think this would be a really not-great thing to make happen every minute in the background, given the burden that VE already exhibits on some users' machines.

Not sure that there is going to be an overall additional burden to the
servers either. Converting HTML RDFa to wikitext will be a memory&processor
hit to the app servers, whereas persisting HTML RDFa into the database is an
IO hit as compared to wikitext.

Sure. But I'm not talking about persisting to the servers, as explained.

Besides, what percentage of overall server load is spent on processing page
saves to content pages (excluding rendering as autosave in VE doesnt need a
render hit)? I'm guessing that isn't where the WMF resources are being
pinched.

By definition, 100% of the Parsoid cluster's load is spent on processing page saves to re-render them now they have changed, either caused by a Parsoid client (like VE) or inside MW from another source (like the wikitext editor). The split is currently weighted towards MW changes, but that's an artefact of the current proportion of edits done with VisualEditor; we expect that figure to rise over time towards 100%.

• We would want (but not strictly /need/) something very flexible, so we

could persist undo stacks, etc. which could add a huge amount of
complexity to the Drafts extension for just one use case/client.

I agree with Legoktm; it would be great if the source editor also had some
features added to it. undo stacks is a major part of the problems holding up
the deployment of Drafts; Drafts needs to handle multiple revisions of the
same page being saved as a draft. I hear the majority of the editor base
tend to use the source editor most of the time ;-)

Sure. But who's going to do that? Building expectations for features on the never-never isn't great for editors. :-(

James' answer above is very authoritative and the concerns about server-side private drafts apply whether we're talking about wikitext or about VE.

I like the general principle of using client-side storage for data recovery purposes (maximum privacy, avoids legal issues), and working towards better support for public drafts (inviting collaborators etc.) in future. That approach is more consistent with wiki-style collaboration in public while also safeguarding user actions.

Thanks for the explanation of the legal issues relating to private drafts. I've created bug T57670 as a result.(In reply to comment #47)

Regarding the client-side hit of a VE->Drafts interface vs VE->Local storage, I suppose I was expecting that VE would be putting HTML RDFa into local storage, in which case the client side hit is the same for both. But that should be discussed on bug T57370. (I've raised a few other points to consider there)

In T39992#424332, @Jdforrester-WMF wrote:

If we create a server-side private drafts facility, we invite malefactors to (ab)use the facility to store and (using accounts with a shared password) distribute:

• Terrorism-related materials;
• Child pornography (even if the Drafts are 'just' text, e.g. base64 encoding of images);
• Foreign intelligence service / espionage channels (let's please not make the Wikileaks comparisons true);
• Other criminal information; and
• Civil illicit information (e.g. breaking NDAs, corporate espionage, …)

All this kind of stuff is potentially uploaded to the public fora of Wikimedia's wikis (talk pages, etc.), but (because people are aware that it's public) it is much less frequently put there, and so is at a level where it can generally be dealt with by the existing community processes. A private cache of drafts that isn't visible to any other user means that the community couldn't police it, or flag for suppression/other admin action.

Because of this, *even if no terrorist/pædophile/spy/criminal/mole ever did this*, WMF would get many, many subpœnæ, warrants, clandestine requests (National Security Letters, etc.) and other legal instruments which would have the effect of hugely increasing the cost to comply with these matters. I appreciate that lots of the other top-10 websites have server-side drafts facilities, but they have lots more money to throw away like this, and (allegedly) direct NSA/FBI/… access to their servers anyway, reducing the legal compliance cost (but something to which we would never agree, of course). People running less-visible services run into this problem from time to time, and in some cases end up shutting down their services; we can't take the risk that we'd need to do this at Wikimedia.

This is the classic argument from infocalypse [1] which is bad form in general, but makes even less sense in Wikimedia's case given that storing arbitrary private data is already possible via the options API [2]. In fact, any user with some Javascript skills could implement the Drafts extension right now as a user script. Even if arbitrary-key, arbitrary-length options got disabled, there would be ways to make Wikipedia store private information (e.g. split it into 255-byte chunks and store it as signatures over a set of autogenerated user accounts).

Not that there is any need of that, given that Wikipedia can be edited by anyone and edits are practically never deleted, so a terrorist/pedophile/etc. who is particularly set on using Wikipedia to distribute illegal content could easily do that with some industry-grade cryptography tool and a bot that hides secret messages in vandalism. Or encode messages via steganography in penis selfies, etc.

Not that there is any need of *that*, given that other WMF-operated services give you easier ways of sharing secret messages, such as etherpad which is essentially private if you choose a pad name that's hard enough to guess.

Not that there is any need of that either, of course, since the net is full of services which, unlike Wikipedia, can be used easily and conveniently using pre-existing software to share information in secret. Dropbox, Google Drive etc, and we didn't even get to tools which were designed explicitly for that purpose like Freenet. So I find the notion that anyone would choose to use Wikipedia's draft feature as their underground distribution channel somewhat absurd.

I appreciate the argument that such activities don't actually need to happen for law enforcement to decide that they need to police them, but as I said, they are already possible, so Drafts would not change the status quo at all.

[1] https://en.wikipedia.org/wiki/Four_Horsemen_of_the_Infocalypse
[2] https://www.mediawiki.org/wiki/API:Options

Legal issues put aside, using client-side storage for drafts does have a lot of advantages: better performance, support for anonymous edits, ability to recover from crashes that happen while there is no network connection. The main disadvantage that I can see is that it is not privacy-friendly - using good privacy practices like incognito mode of the browser would cripple a client-side draft feature as it could not recover the text after a browser crash.

About ability to recover from crashes, I wonder how complex would it be enable this "cache" that Phabricator offers in this very Comments textarea I'm typing it. Type something, reboot your laptop, come back to this URL, and your text is still here, in the textarea, where you left it.

I'd generally prefer client-side drafts, but as discussed offline with @Jdforrester-WMF there is no blanket prohibition from the legal team on server-side draft storage.

Is this still relevant now that Edit Recovery is a thing?