P129 might be useful

In T822#832125, @Gilles wrote:

P129 might be useful

If I calculated correctly, this script cloned about 875 Multimedia Mingle cards as Phabricator tasks in about 4 hours, and without any service interruption.

The only problem that I have heard is that nobody was really aware that this was going to happen, and thousands of email notifications could have been avoided, surely saving 1000s of email to the Multimedia team members, and perhaps reducing the lag this caused in the email notifications queue. Still, to me personally this sounds like a not too bad price to pay for such great result.

@Arrbee, is the Language-Engineering team interested in using this script to clone your cards?

About the Fundraising Tech team, @atgo says that for now they are not thinking of migrating their Mingle cards. We are meeting this week to discuss their next steps.

@Awjrichards, does Tesm Practices have cards in Mingle to clone in Phabricator?

Any other team using Mingle?

Hi Quim, We are definitely interested. But I'd like to know a bit more about:

• what cards the script will migrate i.e. all or only the open cards?
• will all types of cards be migrated? (we had a few custom card types)
• do we need to devote engineering cycles to customize the script? (I believe gilles mentioned that large parts of the script will have to be rewritten for other teams to use)

Thanks.

In T822#833372, @Qgil wrote:

In T822#832125, @Gilles wrote:

P129 might be useful

The only problem that I have heard is that nobody was really aware that this was going to happen, and thousands of email notifications could have been avoided, surely saving 1000s of email to the Multimedia team members, and perhaps reducing the lag this caused in the email notifications queue. Still, to me personally this sounds like a not too bad price to pay for such great result.

Based on the fact that I had done a bunch of batch editing on hundreds of tickets last week, using built-in phabricator tools, which also generates a hundreds of emails and that didn't cause trouble beyond the IRC bot being loud because nobody was around to mute it, it didn't seem any different. Phabricator offers powerful tools right out of the box to edit large amount of tasks and there is no difference between how large of an email burst you can cause through the UI or through the API. If you're worrying about large quantities of emails, worry about the tools that are available in the UI, anyone can use them and send hundreds of emails in a few clicks. It's a core feature of phabricator.

Personally I think that the notifications/emails are useful unmuted, because for the average user who's subscribed to a few tasks or an extension, knowing about the updates is important. As for my fellow team members who received all the emails, they didn't complain to me. I wouldn't expect them to, getting rid of Mingle entirely far outweighs the email nuisance. The level of the nuisance has to be put into perspective too: a simple email search on the bot's name allows one to mark all emails as read very quickly. We're talking about minutes wasted at most...

As for the infrastructure, it would have been a very bad sign to be unable to create hundreds of cards over the course of a few hours. It's a very small amount of records for any respectable database.

@Gilles, I'm not complaining at all! My only advice for the Language Team is to disable email notifications before the migration and enable them after.

@Arrbee, I can't answer your questions, but maybe Gilles...

@Qgil Team Practices does not have any cards in Mingle. The only other teams I am aware of currently using Mingle are Fundraising Tech and Language engineering.

@Arrbee you'll definitely need to devote some engineering time to customizing the script for the way your team is/was using Mingle.

what cards the script will migrate i.e. all or only the open cards?

Up to you, in our case I migrated everything. The way I could tell whether a card was "closed" in Mingle was whether it was assigned to a specific sprint.

will all types of cards be migrated? (we had a few custom card types)

My script whitelists the types of cards to migrate, we also had special cards that didn't need to be ported over (sprints, cycles, meetings)

@Gilles Thank you. That is very helpful information. Whitelisting the card types will solve a good part of the problems. But I think we may have to look at the script to see if the status value can be tweaked.

@Qgil We may do this during or after the holiday downtime.

@Arrbee @Gilles - I dunno if it would be useful at this point, but I wrote a Mingle API wrapper library in python for use with Bingle - it makes it fairly trivial to connect to the Mingle API to find cards and make arbitrary MQL queries; which means you can use it to fetch whatever cards you can get Mingle to return with an MQL query. You could use the library with a custom execution script which depending on you situation may or may not be easier than retrofitting Gilles' single purpose script.

You can see the lib here: https://github.com/wikimedia/bingle/blob/master/lib/mingle.py

@Awjrichards Thank you. Probably worth exploring while we are at it.

A month later... @Arrbee, any progress?

What happens of the closed cards in mingle if they're not migrated? Is there a long-term preservation strategy for the mingle instances?

I don't know how much is the fee for keeping the current instance open. Does Mingle offer a way to dump, the contents of a project in some kind of tarball?

I guess the team in this situation will maintain the cards for a while just in case they still need them, and after a year or so clean the house, considering that anything not migrated is basically irrelevant for daily access at that point?

In T822#978759, @Qgil wrote:

I don't know how much is the fee for keeping the current instance open.

We should double check with @Eloquence but iirc, Thoughtworks is offering hosted Mingle for free for us.

Does Mingle offer a way to dump, the contents of a project in some kind of tarball?

Yes. There are a few different ways of going about this - one is doing it ourselves (a project admin can dump the entire contents of their project from the 'project admin' view, which I think results in some sort of binary file readable by Mingle), we can ask Thoughtworks to do this for us (I think we've had them export whole projects or bunches of projects before, though I'd have to dig through my old emails for specifics - we can conceivably get different kinds of formats out), or you can do an XML dump of all the cards that you get back from an arbitrary 'MQL' (Mingle Query Language) query from within Mingle itself, or using the API (Bingle as a straightforward API wrapper library in Python for interfacing with Mingle).

I guess the team in this situation will maintain the cards for a while just in case they still need them, and after a year or so clean the house, considering that anything not migrated is basically irrelevant for daily access at that point?

Yeah, I would say this is entirely up to the team in question. I don't think there are any teams still using Mingle (though I'm not sure if that's 100% true). We should perhaps let teams play around with Phabricator for a while longer before pulling the plug on Mingle entirely, but it seems like a good idea to start talking about what we want to do with the Mingle instance.

Fundraising isn't migrating our backlog, but we could migrate closed tasks for record keeping.

As far as keeping Mingle alive, I'd like to keep it for a while (a year seems like plenty of time) for reference. By then anything that hasn't made it over can be dumped for sure. Important things will come back.

Yes, the Mingle instance is free to us. I'll ping if there's any change in that relationship.

In T822#970100, @Qgil wrote:

A month later... @Arrbee, any progress?

@Qgil , We will have to wait until the CX deployments are done. The task is still open: T85694

In T822#985218, @Arrbee wrote:

@Qgil , We will have to wait until the CX deployments are done. The task is still open: T85694

That task is closed now.

In T822#2681662, @Aklapper wrote:

T37#1141491 states all Mingle migrations were done 18 months ago. It looks like Gilles' script in P129 has served its purpose and would serve its purpose if anyone needed/wanted to migrate something again. Hence closing this task as "resolved by P129".

Indeed. Thank you.