Wikidata Import 2024-10-26

From BITPlan Wiki
Revision as of 05:49, 28 October 2024 by Wf (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Import
edit
state  ✅
url  https://wiki.bitplan.com/index.php/Wikidata_Import_2024-10-26
target  QLever
start  2024-10-26
end  2024-10-27
days  0.9
os  Ubuntu 22.04.3 LTS
cpu  Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (16 cores)
ram  512
triples  16.3
comment  latest-all.ttl.bz2 23-Oct-2024 17:41 113759486521 ->/hd/beta/qlever/wikidata_20241026/

Using qlv script

see qlv script

# remove failed attempt
rm -rf /hd/delta/qlever/wikidata_20241024/
# check diskspace
qlv -s
Directory  Device           Available      Total Type
alpha      /dev/sdb1             2.0T       3.5T  SSD
beta       /dev/sdc1             2.2T       3.5T  SSD
delta      /dev/sde1             2.8T       3.5T  SSD
eneco      /dev/sda1             8.7T        11T  HDD
gamma      /dev/sdd1             3.2T       3.5T  SSD
mantax     /dev/nvme0n1p1        1.1T       5.8T  SSD
# check choice of disk for today
qlv --today
/hd/beta
# start import
qlv -ir
✅:Created directory /hd/beta/qlever/wikidata_20241026
✅:Started screen session qlever_wikidata_20241026.
✅:Logging to /hd/beta/qlever/wikidata_20241026/screen.log
head -1  /hd/beta/qlever/wikidata_20241026/screen.log
Starting QLever indexing process at Sat Oct 26 06:44:55 AM CEST 2024

tail -f  /hd/beta/qlever/wikidata_20241026/screen.log
eval "$(register-python-argcomplete qlever)" && export QLEVER_ARGCOMPLETE_ENABLED=1


Command: get-data

curl -LRC - --remote-name-all https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2 https://dumps.wikimedia.org/wikidatawiki/entities/latest-lexemes.ttl.bz2 2>&1

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  105G    0  255M    0     0  3859k      0  7:59:43  0:01:07  7:58:36 2054k
QLever indexing process completed at Sun Oct 27 10:31:21 AM CET 2024