Difference between revisions of "Wikidata Import 2024-11-23"

From BITPlan Wiki
Jump to navigation Jump to search
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
{{PageSequence|prev=Wikidata Import 2024-11-16|next=|category=Wikidata|categoryIcon=cloud-download}}
 
{{PageSequence|prev=Wikidata Import 2024-11-16|next=|category=Wikidata|categoryIcon=cloud-download}}
 
{{Import
 
{{Import
−
|state=
+
|state=✅
 
|url=https://wiki.bitplan.com/index.php/Wikidata_Import_2024-11-23
 
|url=https://wiki.bitplan.com/index.php/Wikidata_Import_2024-11-23
 
|target=QLever
 
|target=QLever
 
|start=2024-11-23
 
|start=2024-11-23
−
|end=
+
|end=2024-11-24
−
|days=
+
|days=0.7
 
|os=Ubuntu 22.04.3 LTS
 
|os=Ubuntu 22.04.3 LTS
 
|cpu=Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (16 cores)
 
|cpu=Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (16 cores)
Line 14: Line 14:
 
|storemode=property
 
|storemode=property
 
}}
 
}}
 +
== Update qlever ==
 +
=== Docker images ===
 
<source lang='bash' highlight='1'>
 
<source lang='bash' highlight='1'>
 
qlv -p
 
qlv -p
Line 37: Line 39:
 
docker.io/adfreiburg/qlever-ui:latest
 
docker.io/adfreiburg/qlever-ui:latest
 
✅:Successfully pulled adfreiburg/qlever-ui
 
✅:Successfully pulled adfreiburg/qlever-ui
 +
</source>
 +
=== Qlever Control ===
 +
<source lang='bash' highlight='1,27'>
 +
qlv -qc
 +
Setting up QLever control in /opt/qlever-control...
 +
Cloning into '/opt/qlever-control'...
 +
remote: Enumerating objects: 2074, done.
 +
remote: Counting objects: 100% (929/929), done.
 +
remote: Compressing objects: 100% (292/292), done.
 +
remote: Total 2074 (delta 556), reused 777 (delta 509), pack-reused 1145 (from 1)
 +
Receiving objects: 100% (2074/2074), 501.46 KiB | 6.96 MiB/s, done.
 +
Resolving deltas: 100% (1108/1108), done.
 +
error: pathspec 'python-qlever' did not match any file(s) known to git
 +
Processing /opt/qlever-control
 +
  Installing build dependencies ... done
 +
  Getting requirements to build wheel ... done
 +
  Preparing metadata (pyproject.toml) ... done
 +
Building wheels for collected packages: UNKNOWN
 +
  Building wheel for UNKNOWN (pyproject.toml) ... done
 +
  Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=5111 sha256=62adab49b48044fdda07cbb275fa4d11d9b5713ec0eeae883ff7d2d5070d8a2f
 +
  Stored in directory: /root/.cache/pip/wheels/e5/cd/6c/cbe6881bcd0490208d9bc2c9eb1e1f577f3b753b7e33f9e035
 +
Successfully built UNKNOWN
 +
Installing collected packages: UNKNOWN
 +
Successfully installed UNKNOWN-0.0.0
 +
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
 +
✅:QLever control setup and installed successfully.
 +
✅:QLever is installed and available at /home/wf/.local/bin/qlever
 +
wf@wikidata:/opt$ qlever --version
 +
qlever 0.5.3
 +
</source>
 +
= Start indexing =
 +
<source lang='bash' highlight='1,5,12'>
 +
qlv --disk alpha --index-run
 +
✅:Created directory /hd/alpha/qlever/wikidata_20241123
 +
✅:Started screen session qlever_wikidata_20241123.
 +
✅:Logging to /hd/alpha/qlever/wikidata_20241123/screen.log
 +
tail -f screen.log
 +
QLever server already running on port 7001
 +
To kill the existing server, use `qlever top` or `qlever start` with option --kill-existing-with-same-port`
 +
Show all processes on this machine where the command line matches ^ServerMain.* -p *7001 using Python's psutil library
 +
PID      USER    START    RSS COMMAND
 +
5282    wf      Nov15    4G ServerMain -i wikidata -j 8 -p 7001 -m 20G -c 10G -e 1G -k 200 -s 30s -a wikidata_GtdRwNQw9y4x
 +
QLever indexing process completed at Sun Nov 24 02:03:23 AM CET 2024
 +
/hd/alpha/qlever/wikidata_20241123$ qlever start --kill-existing-with-same-port
 
</source>
 
</source>

Latest revision as of 08:10, 26 November 2024

Import
edit
state  ✅
url  https://wiki.bitplan.com/index.php/Wikidata_Import_2024-11-23
target  QLever
start  2024-11-23
end  2024-11-24
days  0.7
os  Ubuntu 22.04.3 LTS
cpu  Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (16 cores)
ram  512
triples  16.3
comment  latest-all.ttl.bz2 20-Nov-2024 16:16 113740530733->/hd/alpa/qlever/wikidata_20241123/

Update qlever

Docker images

qlv -p
Pulling QLever Docker images...
Using default tag: latest
latest: Pulling from adfreiburg/qlever
6414378b6477: Already exists 
866b6a3b9341: Pull complete 
7d24fb5d8d82: Pull complete 
10eee5628c6d: Pull complete 
f83c68985b10: Pull complete 
0fc8128c9e7c: Pull complete 
ec958d08e409: Pull complete 
cf2eb759a21a: Pull complete 
Digest: sha256:f3bc667462103a1b03c5f1796ca3b61486b67ef8357024aebf21b67606be8c20
Status: Downloaded newer image for adfreiburg/qlever:latest
docker.io/adfreiburg/qlever:latest
✅:Successfully pulled adfreiburg/qlever
Using default tag: latest
latest: Pulling from adfreiburg/qlever-ui
Digest: sha256:f15c908b8db15695f2dde8a841eff29c02ba4dc0c42e08a2f46049613138097e
Status: Image is up to date for adfreiburg/qlever-ui:latest
docker.io/adfreiburg/qlever-ui:latest
✅:Successfully pulled adfreiburg/qlever-ui

Qlever Control

qlv -qc
Setting up QLever control in /opt/qlever-control...
Cloning into '/opt/qlever-control'...
remote: Enumerating objects: 2074, done.
remote: Counting objects: 100% (929/929), done.
remote: Compressing objects: 100% (292/292), done.
remote: Total 2074 (delta 556), reused 777 (delta 509), pack-reused 1145 (from 1)
Receiving objects: 100% (2074/2074), 501.46 KiB | 6.96 MiB/s, done.
Resolving deltas: 100% (1108/1108), done.
error: pathspec 'python-qlever' did not match any file(s) known to git
Processing /opt/qlever-control
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: UNKNOWN
  Building wheel for UNKNOWN (pyproject.toml) ... done
  Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=5111 sha256=62adab49b48044fdda07cbb275fa4d11d9b5713ec0eeae883ff7d2d5070d8a2f
  Stored in directory: /root/.cache/pip/wheels/e5/cd/6c/cbe6881bcd0490208d9bc2c9eb1e1f577f3b753b7e33f9e035
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-0.0.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
✅:QLever control setup and installed successfully.
✅:QLever is installed and available at /home/wf/.local/bin/qlever
wf@wikidata:/opt$ qlever --version
qlever 0.5.3

Start indexing

qlv --disk alpha --index-run
✅:Created directory /hd/alpha/qlever/wikidata_20241123
✅:Started screen session qlever_wikidata_20241123.
✅:Logging to /hd/alpha/qlever/wikidata_20241123/screen.log
tail -f screen.log 
QLever server already running on port 7001
To kill the existing server, use `qlever top` or `qlever start` with option --kill-existing-with-same-port`
Show all processes on this machine where the command line matches ^ServerMain.* -p *7001 using Python's psutil library
PID      USER     START    RSS COMMAND
5282     wf       Nov15     4G ServerMain -i wikidata -j 8 -p 7001 -m 20G -c 10G -e 1G -k 200 -s 30s -a wikidata_GtdRwNQw9y4x
QLever indexing process completed at Sun Nov 24 02:03:23 AM CET 2024
/hd/alpha/qlever/wikidata_20241123$ qlever start --kill-existing-with-same-port
Retrieved from "https://wiki.bitplan.com/index.php?title=Wikidata_Import_2024-11-23&oldid=19373"