Difference between revisions of "Wikidata Import 2018-01-05"
Jump to navigation
Jump to search
(Created page with "{{Import }} = First Attempt 2018-01 = The start of this attempt was on 2018-01-05. I tried to follow the procedure at: * https://github.com/wikimedia/wikidata-query-rdf/blob...") |
|||
| (4 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | =Import= | ||
| + | |||
{{Import | {{Import | ||
| + | |state=✅ | ||
| + | |url=https://wiki.bitplan.com/index.php/Wikidata_Import_2018-01-05 | ||
| + | |target=blazegraph | ||
| + | |start=2018-01-05 | ||
| + | |cpu=Quad-Core AMD Opteron(tm) Processor 2374 HE | ||
}} | }} | ||
| + | =Freitext= | ||
= First Attempt 2018-01 = | = First Attempt 2018-01 = | ||
The start of this attempt was on 2018-01-05. | The start of this attempt was on 2018-01-05. | ||
Latest revision as of 15:24, 14 May 2023
Import
| Import | |
|---|---|
| state | ✅ |
| url | https://wiki.bitplan.com/index.php/Wikidata_Import_2018-01-05 |
| target | blazegraph |
| start | 2018-01-05 |
| end | |
| days | |
| os | |
| cpu | Quad-Core AMD Opteron(tm) Processor 2374 HE |
| ram | |
| triples | |
| comment | |
Freitext
First Attempt 2018-01
The start of this attempt was on 2018-01-05.
I tried to follow the procedure at:
~/wikidata/wikidata-query-rdf/dist/target/service-0.3.0-SNAPSHOT$nohup ./munge.sh -f data/latest-all.ttl.gz -d data/split -l en,de &
#logback.classic pattern: %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
08:23:02.391 [main] INFO org.wikidata.query.rdf.tool.Munge - Switching to data/split/wikidump-000000001.ttl.gz
08:24:21.249 [main] INFO org.wikidata.query.rdf.tool.Munge - Processed 10000 entities at (105, 47, 33)
08:25:07.369 [main] INFO org.wikidata.query.rdf.tool.Munge - Processed 20000 entities at (162, 70, 41)
08:25:56.862 [main] INFO org.wikidata.query.rdf.tool.Munge - Processed 30000 entities at (186, 91, 50)
08:26:43.594 [main] INFO org.wikidata.query.rdf.tool.Munge - Processed 40000 entities at (203, 109, 59)
08:27:24.042 [main] INFO org.wikidata.query.rdf.tool.Munge - Processed 50000 entities at (224, 126, 67)
...
java.nio.file.NoSuchFileException: ./mwservices.json
Import issues
- https://phabricator.wikimedia.org/T164773
- https://phabricator.wikimedia.org/p/Yurik/
- https://www.mediawiki.org/wiki/User:AKlapper_(WMF)
Success
With the use of a 512 GByte SSD disk and carefully monitoring the progress of the import the import succeeded after some 3.8 days.
Queries after import
Number of Triples
SELECT (COUNT(*) as ?Triples) WHERE { ?s ?p ?o}
Triples
3.019.914.549
try it on original WikiData Query Service! Result as of 2020-07-17
Triples 11308353390
Result as of 2022-01-28
13.598.333.948
TypeCount
SELECT ?type (COUNT(?type) AS ?typecount)
WHERE {
?subject a ?type.
}
GROUP by ?type
ORDER by desc(?typecount)
LIMIT 7
<http://wikiba.se/ontology#BestRank> 369637917
schema:Article 61229687
<http://wikiba.se/ontology#GlobecoordinateValue> 5379022
<http://wikiba.se/ontology#QuantityValue> 697187
<http://wikiba.se/ontology#TimeValue> 234556
<http://wikiba.se/ontology#GeoAutoPrecision> 101897
<http://www.wikidata.org/prop/novalue/P17> 37884