Page MenuHomePhabricator
Create Task
Maniphest T239791

DB: perform rolling restart of mariadb daemons to pick up CA changes
Closed, ResolvedPublic
Actions

Description

This ticket is to track progress of the mariadb restarts
The current certificate expires Jun 29 19:36:29 2020 GMT

  • es1012: es1
  • es1016: es1
  • es1018: es1
  • es2011: es1
  • es2012: es1
  • es2013: es1
  • es1011: es2
  • es1013: es2
  • es1015: es2
  • es2014: es2
  • es2015: es2
  • es2016: es2
  • es1014: es3
  • es1017: es3
  • es1019: es3
  • es2017: es3
  • es2018: es3
  • es2019: es3
  • db1117:3321: m1
  • db1135: m1
  • db2078:3321: m1
  • db2132: m1
  • db1117:3322: m2
  • db1132: m2
  • db2078:3322: m2
  • db2133: m2
  • db1117:3323: m3
  • db1128: m3
  • db2078:3323: m3
  • db2134: m3
  • db1107: test-s1
  • db1108: m4
  • db1117:3325: m5
  • db1133: m5
  • db2078:3325: m5
  • db2135: m5
  • pc1007: pc1
  • pc1010: pc1
  • pc2007: pc1
  • pc2010: pc1
  • pc1008: pc2
  • pc2008: pc2
  • pc1009: pc3
  • pc2009: pc3
  • db1080: s1
  • db1083: s1
  • db1089: s1
  • db1099:3311: s1
  • db1105:3311: s1
  • db1106: s1
  • db1118: s1
  • db1119: s1
  • db1107:s1
  • db1114:s1
  • db1124:3311: s1
  • db1134: s1
  • db1139:3311: s1
  • db2071: s1
  • db2072: s1
  • db2085:3311: s1
  • db2088:3311: s1
  • db2092: s1
  • db2094:3311: s1
  • db2097:3311: s1
  • db2103: s1
  • db2112: s1
  • db2116: s1
  • db2130: s1
  • dbstore1003:3311: s1
  • labsdb1009: s1
  • labsdb1010: s1
  • labsdb1011: s1
  • labsdb1012: s1
  • db1074: s2
  • db1076: s2
  • db1090:3312: s2
  • db1095:3312: s2
  • db1103:3312: s2
  • db1105:3312: s2
  • db1122: s2
  • db1125:3312: s2
  • db1129: s2
  • db2088:3312: s2
  • db2091:3312: s2
  • db2095:3312: s2
  • db2098:3312: s2
  • db2104: s2
  • db2107: s2
  • db2108: s2
  • db2125: s2
  • db2126: s2
  • dbstore1004:3312: s2
  • labsdb1009: s2
  • labsdb1010: s2
  • labsdb1011: s2
  • labsdb1012: s2
  • db1075: s3
  • db1078: s3
  • db1095:3313: s3
  • db1112: s3
  • db1123: s3
  • db1124:3313: s3
  • db2074: s3
  • db2094:3313: s3
  • db2098:3313: s3
  • db2105: s3
  • db2109: s3
  • db2127: s3
  • dbstore1004:3313: s3
  • labsdb1009: s3
  • labsdb1010: s3
  • labsdb1011: s3
  • labsdb1012: s3
  • db1081: s4
  • db1084: s4
  • db1091: s4
  • db1097:3314: s4
  • db1102:3314: s4
  • db1103:3314: s4
  • db1121: s4
  • db1125:3314: s4
  • db1138: s4
  • db2073: s4
  • db2084:3314: s4
  • db2090: s4
  • db2091:3314: s4
  • db2095:3314: s4
  • db2099:3314: s4
  • db2106: s4
  • db2110: s4
  • db2119: s4
  • dbstore1004:3314: s4
  • labsdb1009: s4
  • labsdb1010: s4
  • labsdb1011: s4
  • labsdb1012: s4
  • db1082: s5
  • db1096:3315: s5
  • db1097:3315: s5
  • db1100: s5
  • db1102:3315: s5
  • db1110: s5
  • db1113:3315: s5
  • db1124:3315: s5
  • db1130: s5
  • db2075: s5
  • db2084:3315: s5
  • db2089:3315: s5
  • db2094:3315: s5
  • db2099:3315: s5
  • db2111: s5
  • db2113: s5
  • db2123: s5
  • db2128: s5
  • dbstore1003:3315: s5
  • labsdb1009: s5
  • labsdb1010: s5
  • labsdb1011: s5
  • labsdb1012: s5
  • db1085: s6
  • db1088: s6
  • db1093: s6
  • db1096:3316: s6
  • db1098:3316: s6
  • db1113:3316: s6
  • db1125:3316: s6
  • db1131: s6
  • db1139:3316: s6
  • db2076: s6
  • db2087:3316: s6
  • db2089:3316: s6
  • db2095:3316: s6
  • db2097:3316: s6
  • db2114: s6
  • db2117: s6
  • db2124: s6
  • db2129: s6
  • dbstore1005:3316: s6
  • labsdb1009: s6
  • labsdb1010: s6
  • labsdb1011: s6
  • labsdb1012: s6
  • db1079: s7
  • db1086: s7
  • db1090:3317: s7
  • db1094: s7
  • db1098:3317: s7
  • db1101:3317: s7
  • db1116:3317: s7
  • db1125:3317: s7
  • db1136: s7
  • db2077: s7
  • db2086:3317: s7
  • db2087:3317: s7
  • db2095:3317: s7
  • db2100:3317: s7
  • db2118: s7
  • db2120: s7
  • db2121: s7
  • db2122: s7
  • dbstore1003:3317: s7
  • labsdb1009: s7
  • labsdb1010: s7
  • labsdb1011: s7
  • labsdb1012: s7
  • db1087: s8
  • db1092: s8
  • db1099:3318: s8
  • db1101:3318: s8
  • db1104: s8
  • db1109: s8
  • db1116:3318: s8
  • db1124:3318: s8
  • db1126: s8
  • db2079: s8
  • db2080: s8
  • db2081: s8
  • db2082: s8
  • db2083: s8
  • db2085:3318: s8
  • db2086:3318: s8
  • db2094:3318: s8
  • db2100:3318: s8
  • dbstore1005:3318: s8
  • labsdb1009: s8
  • labsdb1010: s8
  • labsdb1011: s8
  • labsdb1012: s8
  • dbstore1005:3350: staging
  • db1115: tendril
  • db2093: tendril
  • db1114: test-s1
  • db2102: test-s1
  • db1077: test-s4
  • db1111: test-s4
  • db1120: x1
  • db1127: x1
  • db1137: x1
  • db1140:3320: x1
  • db2096: x1
  • db2101:3320: x1
  • db2115: x1
  • db2131: x1
  • dbstore1005:3320: x1

Query: mysql.py -hdb1115 zarcillo -e "select instances.name ,section_instances.section from instances join section_instances on instances.name = section_instances.instance order by section_instances.section;

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedjbondT236277 Extend Puppet CA Expiry date
ResolvedjbondT237259 Document all uses of the puppetCA certificate
 ResolvedRLazarusT237362 Rolling restart of etcd to pick up the renewed CA public certificate.
 ResolvedMarosteguiT239791 DB: perform rolling restart of mariadb daemons to pick up CA changes
 ResolvedMarosteguiT244209 Upgrade and restart m5 master (db1133)
 ResolvedMarosteguiT244238 Upgrade and restart m1 master (db1135)
 ResolvedMarosteguiT244566 Upgrade and restart m3 (phabricator) master (db1128)
 ResolvedMarosteguiT246098 Upgrade and restart m2 primary database master (db1132)
 ResolvedTrizek-WMFT250647 Read only windows for database primary masters
 ResolvedMarosteguiT250701 Restart extension1 (x1) database primary master (db1120)
 ResolvedTrizek-WMFT250733 Read only time for extension 1 (x1) primary database on 2020-04-30
 ResolvedMarosteguiT251154 Upgrade and restart s5 and s6 primary DB master: Tue 5th May
 ResolvedTrizek-WMFT251153 Read only time window needed for wikis on s5 and s6
 ResolvedMarosteguiT251158 Upgrade and restart s3 and s7 primary DB master: Thu 7th May
 ResolvedTrizek-WMFT251157 Read only time window needed for wikis on s3 and s7
 ResolvedMarosteguiT251502 Upgrade and restart s4 (commonswiki) primary database master: Tue 12th May
 ResolvedTrizek-WMFT251501 Read only time window needed for Commonswiki
 ResolvedMarosteguiT251981 Upgrade and restart s2 and s8 (wikidatawiki) primary database masters: Tue 19th May
 ResolvedMarosteguiT248086 Drop wb_terms in production from s4 (commonswiki, testcommonswiki), s3 (testwikidatawiki), s8 (wikidatawiki)
 ResolvedJAllemandouT249319 Remove wb_terms from sqoop
 Resolved• BstormT251598 Clean up wb_terms related views
 ResolvedTrizek-WMFT251984 Read only time window needed for s2 wikis and s8 (wikidatawiki)
 ResolvedMarosteguiT251982 Upgrade and restart s1 (enwiki) primary database master: Thu 21th May
 ResolvedTrizek-WMFT251985 Read only time window needed for enwiki
 ResolvedjbondT240330 Check if traffic servers need restarting/reloading post CA change

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Marostegui mentioned this in T244238: Upgrade and restart m1 master (db1135).Feb 4 2020, 2:23 PM
Marostegui renamed this task from DB: perform rolling restart of mariadb deamons to pick up CA changes to DB: perform rolling restart of mariadb daemons to pick up CA changes.Feb 7 2020, 8:57 AM
Marostegui updated the task description. (Show Details)
Marostegui mentioned this in T244566: Upgrade and restart m3 (phabricator) master (db1128).Feb 7 2020, 9:05 AM
Marostegui closed subtask T244209: Upgrade and restart m5 master (db1133) as Resolved.Feb 10 2020, 3:07 PM
Marostegui updated the task description. (Show Details)
Stashbot added a comment.Feb 11 2020, 7:43 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-11T07:43:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1013 - T239791', diff saved to https://phabricator.wikimedia.org/P10374 and previous config saved to /var/cache/conftool/dbconfig/20200211-074358-marostegui.json

Stashbot added a comment.Feb 11 2020, 7:47 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-11T07:47:40Z] <marostegui> Upgrade es1013 - T239791

Marostegui updated the task description. (Show Details)Feb 11 2020, 7:51 AM
Marostegui updated the task description. (Show Details)Feb 11 2020, 8:35 AM
Marostegui closed subtask T244566: Upgrade and restart m3 (phabricator) master (db1128) as Resolved.Feb 13 2020, 6:10 AM
Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)Feb 13 2020, 6:43 AM
Marostegui closed subtask T244238: Upgrade and restart m1 master (db1135) as Resolved.Feb 20 2020, 9:08 AM
Marostegui updated the task description. (Show Details)
Marostegui mentioned this in T246098: Upgrade and restart m2 primary database master (db1132).Feb 25 2020, 12:45 PM
Stashbot added a comment.Mar 2 2020, 9:19 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T09:19:47Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1119 T239791', diff saved to https://phabricator.wikimedia.org/P10569 and previous config saved to /var/cache/conftool/dbconfig/20200302-091947-marostegui.json

Marostegui updated the task description. (Show Details)Mar 2 2020, 9:26 AM
Stashbot added a comment.Mar 2 2020, 9:27 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T09:27:44Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1119 after upgrade T239791', diff saved to https://phabricator.wikimedia.org/P10570 and previous config saved to /var/cache/conftool/dbconfig/20200302-092743-marostegui.json

Stashbot added a comment.Mar 2 2020, 9:38 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T09:38:49Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1119 after upgrade T239791', diff saved to https://phabricator.wikimedia.org/P10572 and previous config saved to /var/cache/conftool/dbconfig/20200302-093848-marostegui.json

Stashbot added a comment.Mar 2 2020, 9:46 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T09:46:34Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1119 after upgrade T239791', diff saved to https://phabricator.wikimedia.org/P10573 and previous config saved to /var/cache/conftool/dbconfig/20200302-094633-marostegui.json

Stashbot added a comment.Mar 2 2020, 9:58 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T09:58:43Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1119 after upgrade T239791', diff saved to https://phabricator.wikimedia.org/P10574 and previous config saved to /var/cache/conftool/dbconfig/20200302-095841-marostegui.json

Stashbot added a comment.Mar 6 2020, 8:44 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:44:47Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1113:3315, db1113:3316 for upgrade - T239791', diff saved to https://phabricator.wikimedia.org/P10640 and previous config saved to /var/cache/conftool/dbconfig/20200306-084439-marostegui.json

Stashbot added a comment.Mar 6 2020, 8:47 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:47:51Z] <marostegui> Stop mysql for db1113:3315, db1113:3316 for upgrade T239791

Marostegui updated the task description. (Show Details)Mar 6 2020, 8:52 AM
Stashbot added a comment.Mar 6 2020, 8:53 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:53:33Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1113:3315, db1113:3316 after upgrade - T239791', diff saved to https://phabricator.wikimedia.org/P10641 and previous config saved to /var/cache/conftool/dbconfig/20200306-085332-marostegui.json

Stashbot added a comment.Mar 6 2020, 8:56 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:56:37Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1074 for upgrade T239791', diff saved to https://phabricator.wikimedia.org/P10642 and previous config saved to /var/cache/conftool/dbconfig/20200306-085435-marostegui.json

Stashbot added a comment.Mar 6 2020, 8:56 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-06T08:56:41Z] <marostegui> Stop MySQL on db1074 for upgrade T239791

Marostegui updated the task description. (Show Details)Mar 6 2020, 8:57 AM
Stashbot added a comment.Mar 9 2020, 2:47 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-09T14:47:52Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1121 T239791', diff saved to https://phabricator.wikimedia.org/P10662 and previous config saved to /var/cache/conftool/dbconfig/20200309-144752-marostegui.json

Stashbot added a comment.Mar 9 2020, 2:48 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-09T14:48:24Z] <marostegui> Restart and upgrade mysql on db1121 T239791

Marostegui updated the task description. (Show Details)Mar 9 2020, 2:51 PM
Stashbot added a comment.Mar 9 2020, 2:52 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-09T14:52:32Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1121 T239791', diff saved to https://phabricator.wikimedia.org/P10663 and previous config saved to /var/cache/conftool/dbconfig/20200309-145232-marostegui.json

Marostegui updated the task description. (Show Details)Mar 9 2020, 3:04 PM
Stashbot added a comment.Mar 9 2020, 3:13 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-09T15:13:11Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1016 to es1 master, this is a NOOP T239791', diff saved to https://phabricator.wikimedia.org/P10664 and previous config saved to /var/cache/conftool/dbconfig/20200309-151310-marostegui.json

Marostegui updated the task description. (Show Details)Mar 9 2020, 3:17 PM
Stashbot added a comment.Mar 9 2020, 3:17 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-09T15:17:51Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1012 T239791', diff saved to https://phabricator.wikimedia.org/P10665 and previous config saved to /var/cache/conftool/dbconfig/20200309-151751-marostegui.json

Stashbot added a comment.Mar 9 2020, 3:29 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-09T15:29:23Z] <marostegui> Upgrade mysql on es1012 T239791

Marostegui updated the task description. (Show Details)Mar 9 2020, 3:31 PM
Stashbot added a comment.Mar 10 2020, 8:25 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-10T08:25:53Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1012 back to es1 master, this is a NOOP T239791', diff saved to https://phabricator.wikimedia.org/P10671 and previous config saved to /var/cache/conftool/dbconfig/20200310-082552-marostegui.json

Stashbot added a comment.Mar 16 2020, 9:30 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-16T09:30:49Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1011 to es2 master, this is a NOOP T239791', diff saved to https://phabricator.wikimedia.org/P10700 and previous config saved to /var/cache/conftool/dbconfig/20200316-093048-marostegui.json

Stashbot added a comment.Mar 16 2020, 9:32 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-16T09:32:29Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1015 for upgrade and restart T239791', diff saved to https://phabricator.wikimedia.org/P10701 and previous config saved to /var/cache/conftool/dbconfig/20200316-093228-marostegui.json

Stashbot added a comment.Mar 16 2020, 10:10 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-16T10:10:57Z] <marostegui> Stop mysql for upgrade on es1015 T239791

Marostegui updated the task description. (Show Details)Mar 16 2020, 10:12 AM
Marostegui closed subtask T246098: Upgrade and restart m2 primary database master (db1132) as Resolved.Mar 19 2020, 9:26 AM
Marostegui updated the task description. (Show Details)
Stashbot added a comment.Mar 20 2020, 7:09 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-20T07:09:23Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1014 to es3 master, this is a NOOP T239791', diff saved to https://phabricator.wikimedia.org/P10734 and previous config saved to /var/cache/conftool/dbconfig/20200320-070922-marostegui.json

Stashbot added a comment.Mar 20 2020, 7:09 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-20T07:09:46Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1017 for update T239791', diff saved to https://phabricator.wikimedia.org/P10735 and previous config saved to /var/cache/conftool/dbconfig/20200320-070945-marostegui.json

Stashbot added a comment.Mar 20 2020, 7:26 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-20T07:26:06Z] <marostegui> Restart mysql on es1017 for upgrade - T239791

Marostegui updated the task description. (Show Details)Mar 20 2020, 7:30 AM

Only s1-s8 and x1 masters pending.

Marostegui moved this task from In progress to Blocked external/Not db team on the DBA board.Mar 30 2020, 8:07 AM
Marostegui added a comment.Apr 10 2020, 11:01 AM

The pending hosts

root@cumin1001:/home/marostegui# for i in `mysql.py -hdb1115 -A zarcillo -e "select instance from masters where dc='eqiad' and section like 's%' or section='x1' and dc='eqiad'" -BN`; do echo $i; mysql.py -h$i -e "show global status like 'Ssl_server_not_after'";done
db1083
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | May 31 09:05:24 2023 GMT |
 ---------------------- -------------------------- 
db1122
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | Apr 23 19:01:32 2023 GMT |
 ---------------------- -------------------------- 
db1123
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | Apr 23 19:02:06 2023 GMT |
 ---------------------- -------------------------- 
db1138
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | May  5 08:44:07 2024 GMT |
 ---------------------- -------------------------- 
db1100
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | Jul  2 10:02:41 2023 GMT |
 ---------------------- -------------------------- 
db1131
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | May  5 06:57:53 2024 GMT |
 ---------------------- -------------------------- 
db1086
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | Jul 10 11:20:07 2023 GMT |
 ---------------------- -------------------------- 
db1109
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | Apr 25 09:04:24 2023 GMT |
 ---------------------- -------------------------- 
db1120
 ---------------------- -------------------------- 
| Variable_name        | Value                    |
 ---------------------- -------------------------- 
| Ssl_server_not_after | Jul 24 14:45:28 2023 GMT |
 ---------------------- --------------------------
Marostegui mentioned this in T250701: Restart extension1 (x1) database primary master (db1120).Apr 20 2020, 2:50 PM
Marostegui added a subtask: T250701: Restart extension1 (x1) database primary master (db1120).
Marostegui mentioned this in T251154: Upgrade and restart s5 and s6 primary DB master: Tue 5th May.Apr 27 2020, 3:12 PM
Marostegui mentioned this in T251158: Upgrade and restart s3 and s7 primary DB master: Thu 7th May.Apr 27 2020, 3:19 PM
jcrespo changed the status of subtask T239238: Switchover s8 primary database master db1109 -> db1104 - 2020-09-29 08:00 UTC from Open to Stalled.Apr 29 2020, 7:30 AM
Marostegui moved this task from Blocked external/Not db team to In progress on the DBA board.Apr 29 2020, 8:03 AM
Naylinko subscribed.Apr 30 2020, 4:56 AM
Marostegui closed subtask T250701: Restart extension1 (x1) database primary master (db1120) as Resolved.Apr 30 2020, 5:08 AM
Marostegui updated the task description. (Show Details)
Marostegui mentioned this in T251502: Upgrade and restart s4 (commonswiki) primary database master: Tue 12th May.Apr 30 2020, 9:15 AM
Marostegui closed subtask T251154: Upgrade and restart s5 and s6 primary DB master: Tue 5th May as Resolved.May 5 2020, 5:29 AM
Marostegui updated the task description. (Show Details)
Marostegui mentioned this in T251981: Upgrade and restart s2 and s8 (wikidatawiki) primary database masters: Tue 19th May.May 6 2020, 7:25 AM
Marostegui added a subtask: T251981: Upgrade and restart s2 and s8 (wikidatawiki) primary database masters: Tue 19th May.May 6 2020, 7:32 AM
Marostegui mentioned this in T251982: Upgrade and restart s1 (enwiki) primary database master: Thu 21th May.May 6 2020, 7:34 AM
Marostegui removed a parent task: T243318: FY2020-2021 Q1 codfw -> eqiad switchback.May 6 2020, 10:24 AM
Marostegui closed subtask T251158: Upgrade and restart s3 and s7 primary DB master: Thu 7th May as Resolved.May 7 2020, 5:14 AM
Marostegui updated the task description. (Show Details)
Marostegui closed subtask T251502: Upgrade and restart s4 (commonswiki) primary database master: Tue 12th May as Resolved.May 12 2020, 5:06 AM
Marostegui updated the task description. (Show Details)
Marostegui closed subtask T251981: Upgrade and restart s2 and s8 (wikidatawiki) primary database masters: Tue 19th May as Resolved.May 19 2020, 5:06 AM
Marostegui updated the task description. (Show Details)
Marostegui closed subtask T251982: Upgrade and restart s1 (enwiki) primary database master: Thu 21th May as Resolved.May 21 2020, 5:06 AM
Marostegui updated the task description. (Show Details)
Marostegui closed this task as Resolved.May 21 2020, 5:09 AM

This is all done!

Marostegui closed subtask T250647: Read only windows for database primary masters as Resolved.May 21 2020, 12:33 PM
Marostegui removed a subtask: T239238: Switchover s8 primary database master db1109 -> db1104 - 2020-09-29 08:00 UTC.Jun 29 2020, 4:30 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits