| • alaa_wmde | |
| Mar 25 2019, 9:19 AM |
| F31671113: image.png | |
| Mar 8 2020, 12:12 PM |
| F31666491: image.png | |
| Mar 5 2020, 10:43 AM |
| F31647231: image.png | |
| Feb 27 2020, 6:08 PM |
| F31647226: image.png | |
| Feb 27 2020, 6:08 PM |
| F31502543: image.png | |
| Jan 6 2020, 4:34 PM |
Status:
TODOs:
Mentioned in SAL (#wikimedia-operations) [2020-03-04T13:47:31Z] <addshore> START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 3)
Mentioned in SAL (#wikimedia-operations) [2020-03-04T17:41:01Z] <addshore> stop item term rebuild at Q Q60345318 as I generate more lists (T219123)
I'm generating some new lists to work from to reduce the amount of time the rest of the migration will take.
Identifying "holes" in the tables, where some records exist, but some in other tables have gone missing due to bugs:
addshore@stat1007:~$ analytics-mysql wikidatawiki -e "SELECT DISTINCT wbit_item_id as id FROM wbt_item_terms LEFT JOIN wbt_term_in_lang ON wbit_term_in_lang_id = wbtl_id LEFT JOIN wbt_type ON wbtl_type_id = wby_id LEFT JOIN wbt_text_in_lang ON wbtl_text_in_lang_id = wbxl_id LEFT JOIN wbt_text ON wbxl_text_id = wbx_id WHERE wbx_text IS NULL ORDER BY wbit_item_id ASC;" -N -B > 4march1740-holes-nulls.list
Another one identifying all items that have no records yet (have not been migrated):
addshore@stat1007:~$ cat 4march1740-holes-86000000.sql
SELECT n
FROM
(
SELECT (a.digit (10 * b.digit) (100 * c.digit) (1000 * d.digit) (10000 * e.digit) (100000 * f.digit) (1000000 * g.digit) (10000000 * h.digit)) as n
from (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as a
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as b
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as c
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as d
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as e
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as f
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as g
cross join (select 0 as digit union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) as h
) as a
LEFT JOIN
( SELECT DISTINCT wbit_item_id from wbt_item_terms where wbit_item_id BETWEEN 0 AND 86000000 ) as b
ON wbit_item_id = n
WHERE wbit_item_id IS NULL
AND n BETWEEN 0 AND 86000000
;After these lists have been migrated we will also have to deal with items above 86 million.
When running these queries earlier I identified roughly 26 million more items to pass over
addshore@stat1007:~$ wc -l 4march1143-holes-*.list 19237277 4march1143-holes-87061632.list 6753614 4march1143-holes-nulls.list 25990891 total addshore@stat1007:~$ sort 4march1143-holes-*.list | uniq | wc -l 25990891
Mentioned in SAL (#wikimedia-operations) [2020-03-05T08:50:28Z] <addshore> START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 1 today)
Mentioned in SAL (#wikimedia-operations) [2020-03-05T10:11:06Z] <addshore> START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 2 today)
Change 577209 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Read from the new term store up to Q30 mill everywhere
Change 577211 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Write to the new terms store up to Q 87 million
Archive of the old description as I am now rebuilding ONLY using SQL generated lists.
Hadoop based Rebuild number 3
- 31 January, Q 1.8 million
- 4 February Q 9 million
- 7 February Q 14.5 million
- 17 February Q 37.5 million
- 23 February Q 44.5 million
- 27 February Q 50 million
- 2 March Q54 million
- 4 March Q60 million
Post Hadoop, SQL based rebuild number 3:
- Fill in gaps missed due to only rebuilding from a select ID list from hadoop
- 0-1 million - Addshore did on 18th Feb, (4 deleted item holes left) - https://phabricator.wikimedia.org/P10447 - and some other holes re appearing T219123#5896364
- 0-10 million - Addshore did on 20th Feb (8122 item holes left) - https://phabricator.wikimedia.org/P10469
- 10-20 million - Addshore did on 25th Feb (1292 item holes left) - https://phabricator.wikimedia.org/P10517
- 20-30 million - Addshore did on 26th Feb (442k holes) TBA how many are left?
- 30-40 million - Addshore did between 3rd & 4th March
- 40-50 million - Addshore did between 3rd & 4th March
- 50-end ??????
Change 577209 merged by jenkins-bot:
[operations/mediawiki-config@master] Read from the new term store up to Q30 mill everywhere
Mentioned in SAL (#wikimedia-operations) [2020-03-05T11:04:55Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Reading up to Q30M for the new term store everywhere (was Q25M) warm db1126 & db1111 caches (T219123) (duration: 01m 05s)
Mentioned in SAL (#wikimedia-operations) [2020-03-05T11:06:08Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Reading up to Q30M for the new term store everywhere (was Q25M) warm db1126 & db1111 caches (T219123) cache bust (duration: 01m 04s)
Change 577211 merged by jenkins-bot:
[operations/mediawiki-config@master] Write to the new terms store up to Q 87 million
Mentioned in SAL (#wikimedia-operations) [2020-03-05T11:19:26Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Write to new term store up to Q87 million, was 86 (T219123) (duration: 01m 04s)
Mentioned in SAL (#wikimedia-operations) [2020-03-05T11:20:34Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Write to new term store up to Q87 million, was 86 (T219123) cache bust (duration: 01m 03s)
Mentioned in SAL (#wikimedia-operations) [2020-03-05T12:52:42Z] <addshore> START warm cache for db1111 & db1126 for Q30-32 million (100k batch selects, 30s sleep) T219123 (pass 1)
Just added the "rebuild" of 86-87 million to the lists that are currently running.
addshore@mwmaint1002:~/sqlterms$ cat ./6/*part* | wc -l 950622
Change 579213 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Write to the new terms store up to Q 87.5 million
Change 579213 merged by jenkins-bot:
[operations/mediawiki-config@master] Write to the new terms store up to Q 87.5 million
Mentioned in SAL (#wikimedia-operations) [2020-03-12T08:26:33Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Write to new term store up to Q87.5 million, was 87 (T219123) (duration: 01m 12s)
Mentioned in SAL (#wikimedia-operations) [2020-03-12T08:27:45Z] <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Write to new term store up to Q87.5 million, was 87 (T219123) cache bust (duration: 01m 08s)
Mentioned in SAL (#wikimedia-operations) [2020-03-12T08:36:02Z] <addshore> start "rebuild" of Q87 -> 87.5 million for T219123
Change 579326 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set term store to WRITE_BOTH for all of Wikidata
Change 579326 merged by jenkins-bot:
[operations/mediawiki-config@master] Set term store to WRITE_BOTH for all of Wikidata
Mentioned in SAL (#wikimedia-operations) [2020-03-12T18:29:43Z] <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:579326|Set term store to WRITE_BOTH for all of Wikidata (T219123)]] (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T10:02:20Z] <Amir1> start of ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --batch-size=50 --sleep=0 --file=15march2217-holes-nulls.list on screen (T219123)
Change 579908 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Revert "Revert "Set term store to WRITE_BOTH for all of Wikidata""
Change 579908 merged by jenkins-bot:
[operations/mediawiki-config@master] Revert "Revert "Set term store to WRITE_BOTH for all of Wikidata""
Mentioned in SAL (#wikimedia-operations) [2020-03-16T10:43:26Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: "Set term store to WRITE_BOTH for all of Wikidata" (T219123) (duration: 01m 13s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T10:45:09Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: "Set term store to WRITE_BOTH for all of Wikidata" (T219123), take II (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T10:55:44Z] <Amir1> warming up db1026 for up to Q35M for the new term store (T219123)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T11:04:04Z] <Amir1> Warming up InnoDB buffer pool cache in db1111, db1126, db1104, db1092 (T219123)
Change 579913 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set up read new term store up to Q35M
Change 579913 merged by jenkins-bot:
[operations/mediawiki-config@master] Set up read new term store up to Q35M
Mentioned in SAL (#wikimedia-operations) [2020-03-16T12:05:32Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579913|Set up read new term store up to Q35M (T219123)]] (duration: 01m 08s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T12:09:39Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579913|Set up read new term store up to Q35M (T219123)]], take II (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T12:09:53Z] <Amir1> warming up cache for Q35M to Q40M for new term store on db1111, db1126, db1104, db1092 (T219123)
Change 579925 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set up read new term store up to Q40M
Change 579925 merged by jenkins-bot:
[operations/mediawiki-config@master] Set up read new term store up to Q40M
Mentioned in SAL (#wikimedia-operations) [2020-03-16T14:15:02Z] <Amir1> ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --from-id 87500000 --to-id 87767570 --batch-size=10 --sleep=5 (T219123)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T14:18:54Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q40M (T219123)]] (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T14:22:13Z] <Amir1> warming up cache for Q40M to Q50M for new term store on db1111, db1126, db1104, db1092 (T219123)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T14:22:19Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q40M (T219123)]], take II (duration: 01m 06s)
Change 580082 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set up read new term store up to Q50M
Change 580082 merged by jenkins-bot:
[operations/mediawiki-config@master] Set up read new term store up to Q50M
Mentioned in SAL (#wikimedia-operations) [2020-03-16T17:03:36Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q50M (T219123)]] (duration: 01m 06s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T17:06:56Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q50M (T219123)]], take II (duration: 01m 08s)
Mentioned in SAL (#wikimedia-operations) [2020-03-16T17:08:12Z] <Amir1> warming up cache for Q50M to Q60M for new term store on db1111, db1126, db1104, db1092 (T219123)
Change 580285 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set up read new term store up to Q60M
Change 580285 merged by jenkins-bot:
[operations/mediawiki-config@master] Set up read new term store up to Q60M
Mentioned in SAL (#wikimedia-operations) [2020-03-17T09:54:19Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q60M (T219123)]] (duration: 01m 09s)
Mentioned in SAL (#wikimedia-operations) [2020-03-17T09:55:53Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q60M (T219123)]], take II (duration: 01m 05s)
Mentioned in SAL (#wikimedia-operations) [2020-03-17T10:03:03Z] <Amir1> warming up cache for Q60M to Q70M for new term store on db1111, db1126, db1104, db1092 (T219123)
Change 580331 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set up read new term store up to Q70M
Change 580331 merged by jenkins-bot:
[operations/mediawiki-config@master] Set up read new term store up to Q70M
Mentioned in SAL (#wikimedia-operations) [2020-03-17T14:28:00Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q70M (T219123)]] (duration: 01m 10s)
Mentioned in SAL (#wikimedia-operations) [2020-03-17T14:29:39Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q70M (T219123)]], take II (duration: 01m 04s)
Mentioned in SAL (#wikimedia-operations) [2020-03-17T17:52:07Z] <Amir1> warming up cache for Q70M to Q80M for new term store on db1111, db1126, db1104, db1092 (T219123)
Change 580397 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Set up read new term store up to Q80M
Change 580397 merged by jenkins-bot:
[operations/mediawiki-config@master] Set up read new term store up to Q80M
Mentioned in SAL (#wikimedia-operations) [2020-03-17T19:00:28Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q80M (T219123)]] (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-17T19:01:40Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q80M (T219123)]], take II (duration: 01m 06s)
Mentioned in SAL (#wikimedia-operations) [2020-03-17T22:49:25Z] <Amir1> warming up cache for Q80M to Q88M for new term store on db1111, db1126, db1104, db1092 (T219123)
Change 580416 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Read from the new term store everywhere
Change 580603 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] labs: Stop writing to the old term store for wikidatawiki
Change 580603 merged by jenkins-bot:
[operations/mediawiki-config@master] labs: Stop writing to the old term store for wikidatawiki
Mentioned in SAL (#wikimedia-releng) [2020-03-18T03:16:34Z] <Amir1> dropping wb_terms table from wikidatawiki in beta cluster (T219123 T219175 T208425)
Change 580416 merged by jenkins-bot:
[operations/mediawiki-config@master] Read from the new term store everywhere
Mentioned in SAL (#wikimedia-operations) [2020-03-18T10:12:48Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Read from the new term store everywhere (T219123)]] (duration: 01m 08s)
Mentioned in SAL (#wikimedia-operations) [2020-03-18T10:14:44Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Read from the new term store everywhere (T219123)]], take II (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-18T10:31:54Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Read from the new term store everywhere (T219123)]] (duration: 01m 07s)
Mentioned in SAL (#wikimedia-operations) [2020-03-18T10:33:11Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Read from the new term store everywhere (T219123)]], take II (duration: 01m 07s)
