Difference between revisions of "Wikidata Import 2024-01-20"
Jump to navigation
Jump to search
| Line 75: | Line 75: | ||
% Total % Received % Xferd Average Speed Time Time Time Current | % Total % Received % Xferd Average Speed Time Time Time Current | ||
Dload Upload Total Spent Left Speed | Dload Upload Total Spent Left Speed | ||
| − | + | 100 1648M 100 1648M 0 0 13.0M 0 0:02:06 0:02:06 --:--:-- 13.1M | |
| + | Total file size: 1.7 GB | ||
| + | |||
| + | Action "index" | ||
| + | |||
| + | Write value of config variable index.SETTINGS_JSON to file dblp.settings.json | ||
| + | docker run -it --rm -u $(id -u):$(id -g) -v /etc/localtime:/etc/localtime:ro -v $(pwd):/index -w /index --entrypoint bash --name qlever.indexer.dblp adfreiburg/qlever -c 'zcat dblp.ttl.gz | IndexBuilderMain -F ttl -f - -i dblp -s dblp.settings.json --text-words-from-literals | tee dblp.index-log.txt' | ||
| + | |||
| + | 2024-01-20 06:31:08.935 - INFO: QLever IndexBuilder, compiled on Fri Jan 19 21:22:46 UTC 2024 using git hash 95fc20 | ||
| + | 2024-01-20 06:31:08.936 - INFO: You specified the input format: TTL | ||
| + | 2024-01-20 06:31:08.936 - INFO: Processing input triples from /dev/stdin ... | ||
| + | 2024-01-20 06:31:08.936 - INFO: Locale was not specified in settings file, default is en_US | ||
| + | 2024-01-20 06:31:08.936 - INFO: You specified "locale = en_US" and "ignore-punctuation = 0" | ||
| + | 2024-01-20 06:31:08.936 - INFO: You specified "parallel-parsing = true", which enables faster parsing for TTL files that don't include multiline literals with unescaped newline characters and that have newline characters after the end of triples. | ||
| + | 2024-01-20 06:31:08.936 - INFO: You specified "num-triples-per-batch = 1,000,000", choose a lower value if the index builder runs out of memory | ||
| + | 2024-01-20 06:31:08.936 - INFO: Integers that cannot be represented by QLever will throw an exception (this is the default behavior) | ||
</source> | </source> | ||
Revision as of 07:31, 20 January 2024
Docker
docker pull adfreiburg/qlever
Using default tag: latest
latest: Pulling from adfreiburg/qlever
29202e855b20: Pull complete
94ca9f61181f: Pull complete
367bd497f93c: Pull complete
5d9353f3c7b1: Pull complete
fa1b81522802: Pull complete
70be4539455c: Pull complete
d3c042ca662a: Pull complete
Digest: sha256:19106e3606851a1b0a3ca736aa6c3a5246ce48913dbc2a434c4a821c6d0af492
Status: Downloaded newer image for adfreiburg/qlever:latest
docker.io/adfreiburg/qlever:latest
docker pull adfreiburg/qlever-ui
Using default tag: latest
latest: Pulling from adfreiburg/qlever-ui
59bf1c3509f3: Already exists
07a400e93df3: Already exists
64052ee245ef: Already exists
a44d093ad4a5: Already exists
0381087ee065: Already exists
91c88323734b: Pull complete
fdcee6d0309d: Pull complete
e6b2715c1d5d: Pull complete
b9c9f00cb678: Pull complete
3f12ea50b177: Pull complete
Digest: sha256:7f4b358d6a127e512979074de0c6e84f250a37bca46c494d8e04a62844716e48
Status: Downloaded newer image for adfreiburg/qlever-ui:latest
docker.io/adfreiburg/qlever-ui:latest
QLever control
https://github.com/ad-freiburg/qlever-control
git clone https://github.com/ad-freiburg/qlever-control.git
Cloning into 'qlever-control'...
remote: Enumerating objects: 1107, done.
remote: Counting objects: 100% (865/865), done.
remote: Compressing objects: 100% (422/422), done.
remote: Total 1107 (delta 392), reused 781 (delta 374), pack-reused 242
Receiving objects: 100% (1107/1107), 242.20 KiB | 8.97 MiB/s, done.
Resolving deltas: 100% (506/506), done.
cd qlever-control/
git checkout python-qlever
Branch 'python-qlever' set up to track remote branch 'python-qlever' from 'origin'.
Switched to a new branch 'python-qlever'
pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/wf/source/python/qlever-control
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: UNKNOWN
Building wheel for UNKNOWN (pyproject.toml) ... done
Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=5111 sha256=1f79a64c282f532143b3ea16551e4698077fd186ce36aa5c306bf6d08822ca26
Stored in directory: /home/wf/.cache/pip/wheels/07/95/58/79d49197785a6e837569fd3f894d646428d2e272f53582c762
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-0.0.0
dblp warmup test
wf@wikidata:/hd/mantax/qlever/dblp$ qlever setup-config dblp
# ...
wf@wikidata:/hd/mantax/qlever/dblp$ qlever get-data index restart test-query ui
Action "get-data"
curl -LO -C - https://dblp.org/rdf/dblp.ttl.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1648M 100 1648M 0 0 13.0M 0 0:02:06 0:02:06 --:--:-- 13.1M
Total file size: 1.7 GB
Action "index"
Write value of config variable index.SETTINGS_JSON to file dblp.settings.json
docker run -it --rm -u $(id -u):$(id -g) -v /etc/localtime:/etc/localtime:ro -v $(pwd):/index -w /index --entrypoint bash --name qlever.indexer.dblp adfreiburg/qlever -c 'zcat dblp.ttl.gz | IndexBuilderMain -F ttl -f - -i dblp -s dblp.settings.json --text-words-from-literals | tee dblp.index-log.txt'
2024-01-20 06:31:08.935 - INFO: QLever IndexBuilder, compiled on Fri Jan 19 21:22:46 UTC 2024 using git hash 95fc20
2024-01-20 06:31:08.936 - INFO: You specified the input format: TTL
2024-01-20 06:31:08.936 - INFO: Processing input triples from /dev/stdin ...
2024-01-20 06:31:08.936 - INFO: Locale was not specified in settings file, default is en_US
2024-01-20 06:31:08.936 - INFO: You specified "locale = en_US" and "ignore-punctuation = 0"
2024-01-20 06:31:08.936 - INFO: You specified "parallel-parsing = true", which enables faster parsing for TTL files that don't include multiline literals with unescaped newline characters and that have newline characters after the end of triples.
2024-01-20 06:31:08.936 - INFO: You specified "num-triples-per-batch = 1,000,000", choose a lower value if the index builder runs out of memory
2024-01-20 06:31:08.936 - INFO: Integers that cannot be represented by QLever will throw an exception (this is the default behavior)