Difference between revisions of "Wikidata Import 2025-05-02"
Jump to navigation
Jump to search
| Line 23: | Line 23: | ||
== git clone the priv-wd-query == | == git clone the priv-wd-query == | ||
| − | === === | + | <source lang='bash' highlight='1-4,6-7> |
| − | Incompatible RWStore header version | + | git clone https://github.com/scatter-llc/private-wikidata-query |
| + | mkdir data | ||
| + | mv data.jnl private-wikidata-query/data | ||
| + | cd private-wikidata-query/data | ||
| + | # use proper uid and gid as per the containers preferences | ||
| + | chown 666:66 data.jnl | ||
| + | jh@wikidata:/hd/delta/blazegraph/private-wikidata-query/data$ ls -l | ||
| + | total 346081076 | ||
| + | -rw-rw-r-- 1 666 66 1328514809856 May 2 22:07 data.jnl | ||
| + | </source> | ||
| + | === start docker === | ||
| + | <source lang='bash' highlight='1'> | ||
| + | docker compose up -d | ||
| + | WARN[0000] /hd/delta/blazegraph/private-wikidata-query/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion | ||
| + | [+] Running 3/3 | ||
| + | ✔ Container private-wikidata-query-wdqs-1 Started 0.4s | ||
| + | ✔ Container private-wikidata-query-wdqs-proxy-1 Started 0.7s | ||
| + | ✔ Container private-wikidata-query-wdqs-frontend-1 Started 1.1s | ||
| + | </source> | ||
| + | ==== Incompatible RWStore header version ==== | ||
Revision as of 10:55, 3 May 2025
Import
| Import | |
|---|---|
 edit
| |
| state | |
| url | https://wiki.bitplan.com/index.php/Wikidata_Import_2025-05-02 |
| target | blazegraph |
| start | 2025-05-02 |
| end | 2025-05-03 |
| days | 0.6 |
| os | Ubuntu 22.04.3 LTS |
| cpu | Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (16 cores) |
| ram | 512 |
| triples | |
| comment | |
This "import" is not using a dump and indexing approach but directly copying a blazegraph journal file.
Steps
Copy journal file
Source https://scatter.red/ wikidata installation. Usimng aria2c with 16 connections the copy took some 5 hours.
git clone the priv-wd-query
git clone https://github.com/scatter-llc/private-wikidata-query
mkdir data
mv data.jnl private-wikidata-query/data
cd private-wikidata-query/data
# use proper uid and gid as per the containers preferences
chown 666:66 data.jnl
jh@wikidata:/hd/delta/blazegraph/private-wikidata-query/data$ ls -l
total 346081076
-rw-rw-r-- 1 666 66 1328514809856 May 2 22:07 data.jnl
start docker
docker compose up -d
WARN[0000] /hd/delta/blazegraph/private-wikidata-query/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Running 3/3
✔ Container private-wikidata-query-wdqs-1 Started 0.4s
✔ Container private-wikidata-query-wdqs-proxy-1 Started 0.7s
✔ Container private-wikidata-query-wdqs-frontend-1 Started 1.1s