Difference between revisions of "Wikidata Import 2023-04-18"
Jump to navigation
Jump to search
| Line 20: | Line 20: | ||
</pre> | </pre> | ||
see https://addshore.com/2019/10/your-own-wikidata-query-service-with-no-limits/ | see https://addshore.com/2019/10/your-own-wikidata-query-service-with-no-limits/ | ||
| − | == latest-all.ttl.gz ~2 hours = | + | == latest-all.ttl.gz ~2 hours == |
<source lang='bash' highlight='1'> | <source lang='bash' highlight='1'> | ||
nohup wget https://dumps.wikimedia.your.org/wikidatawiki/entities/latest-all.ttl.gz& | nohup wget https://dumps.wikimedia.your.org/wikidatawiki/entities/latest-all.ttl.gz& | ||
| Line 51: | Line 51: | ||
100K .......... .......... .......... .......... .......... 0% 385K 3d18h | 100K .......... .......... .......... .......... .......... 0% 385K 3d18h | ||
</source> | </source> | ||
| + | |||
= Munging = | = Munging = | ||
* https://github.com/wikimedia/operations-cookbooks/blob/master/cookbooks/sre/wdqs/data-reload.py | * https://github.com/wikimedia/operations-cookbooks/blob/master/cookbooks/sre/wdqs/data-reload.py | ||
Revision as of 11:46, 18 April 2023
Download latest wikidata Dump ~2 hours
Download Options
https://dumps.wikimedia.org/wikidatawiki/entities
latest-all.json.bz2 12-Apr-2023 17:28 81311587032 latest-all.json.gz 12-Apr-2023 08:58 123350716043 latest-all.nt.bz2 13-Apr-2023 07:26 157814250243 latest-all.nt.gz 12-Apr-2023 13:38 204367313580 latest-all.ttl.bz2 12-Apr-2023 18:47 101241734022 latest-all.ttl.gz 12-Apr-2023 08:55 123611937673 latest-lexemes.json.bz2 12-Apr-2023 03:53 292437658 latest-lexemes.json.gz 12-Apr-2023 03:51 400418207 latest-lexemes.nt.bz2 14-Apr-2023 23:35 763000897 latest-lexemes.nt.gz 14-Apr-2023 23:29 1002680227 latest-lexemes.ttl.bz2 14-Apr-2023 23:30 430016024 latest-lexemes.ttl.gz 14-Apr-2023 23:26 536802172 latest-truthy.nt.bz2 14-Apr-2023 18:25 35953038205 latest-truthy.nt.gz 14-Apr-2023 15:04 59629575771
see https://addshore.com/2019/10/your-own-wikidata-query-service-with-no-limits/
latest-all.ttl.gz ~2 hours
nohup wget https://dumps.wikimedia.your.org/wikidatawiki/entities/latest-all.ttl.gz&
--2023-04-18 08:27:54-- https://dumps.wikimedia.your.org/wikidatawiki/entities/latest-all.ttl.gz
Resolving dumps.wikimedia.your.org (dumps.wikimedia.your.org)... 163.237.219.7, 2001:4978:1000:1499::7
Connecting to dumps.wikimedia.your.org (dumps.wikimedia.your.org)|163.237.219.7|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 117784845970 (110G) [application/octet-stream]
Saving to: ‘latest-all.ttl.gz’
0K .......... .......... .......... .......... .......... 0% 417K 3d4h
...
115024200K .......... .......... .......... .......... .......... 99% 29.6M 0s
115024250K .......... ... 100% 126M=1h59m
2023-04-18 10:27:37 (15.6 MB/s) - ‘latest-all.ttl.gz’ saved [117784845970/117784845970]
latest-all.ttl.bz2
nohup wget https://dumps.wikimedia.your.org/wikidatawiki/entities/latest-all.ttl.bz2&
--2023-04-18 12:43:54-- https://dumps.wikimedia.your.org/wikidatawiki/entities/latest-all.ttl.bz2
Resolving dumps.wikimedia.your.org (dumps.wikimedia.your.org)... 163.237.219.7, 2001:4978:1000:1499::7
Connecting to dumps.wikimedia.your.org (dumps.wikimedia.your.org)|163.237.219.7|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 96670186177 (90G) [application/octet-stream]
Saving to: ‘latest-all.ttl.bz2’
0K .......... .......... .......... .......... .......... 0% 384K 2d20h
50K .......... .......... .......... .......... .......... 0% 193K 4d5h
100K .......... .......... .......... .......... .......... 0% 385K 3d18h