Difference between revisions of "Wikidata Import 2024-10-25"

From BITPlan Wiki
Jump to navigation Jump to search
(Created page with "{{PageSequence|prev=Wikidata Import 2024-10-24|next=|category=Wikidata|categoryIcon=cloud-download}}")
 
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
{{PageSequence|prev=Wikidata Import 2024-10-24|next=|category=Wikidata|categoryIcon=cloud-download}}
 
{{PageSequence|prev=Wikidata Import 2024-10-24|next=|category=Wikidata|categoryIcon=cloud-download}}
 +
<source lang='bash' highlight="2,4,13,16,20,23">
 +
# remove failed attempt
 +
rm -rf /hd/delta/qlever/wikidata_20241024/
 +
# check diskspace
 +
qlv -s
 +
Directory  Device          Available      Total Type
 +
alpha      /dev/sdb1            2.0T      3.5T  SSD
 +
beta      /dev/sdc1            2.2T      3.5T  SSD
 +
delta      /dev/sde1            2.8T      3.5T  SSD
 +
eneco      /dev/sda1            8.7T        11T  HDD
 +
gamma      /dev/sdd1            3.2T      3.5T  SSD
 +
mantax    /dev/nvme0n1p1        1.1T      5.8T  SSD
 +
# check choice of disk for today
 +
qlv --today
 +
/hd/beta
 +
# start import
 +
qlv -ir
 +
✅:Created directory /hd/beta/qlever/wikidata_20241026
 +
✅:Started screen session qlever_wikidata_20241026.
 +
✅:Logging to /hd/beta/qlever/wikidata_20241026/screen.log
 +
head -1  /hd/beta/qlever/wikidata_20241026/screen.log
 +
Starting QLever indexing process at Sat Oct 26 06:44:55 AM CEST 2024
 +
 +
tail -f  /hd/beta/qlever/wikidata_20241026/screen.log
 +
eval "$(register-python-argcomplete qlever)" && export QLEVER_ARGCOMPLETE_ENABLED=1
 +
 +
 +
Command: get-data
 +
 +
curl -LRC - --remote-name-all https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2 https://dumps.wikimedia.org/wikidatawiki/entities/latest-lexemes.ttl.bz2 2>&1
 +
 +
  % Total    % Received % Xferd  Average Speed  Time    Time    Time  Current
 +
                                Dload  Upload  Total  Spent    Left  Speed
 +
  0  105G    0  255M    0    0  3859k      0  7:59:43  0:01:07  7:58:36 2054k
 +
</source>

Latest revision as of 05:50, 26 October 2024

# remove failed attempt
rm -rf /hd/delta/qlever/wikidata_20241024/
# check diskspace
qlv -s
Directory  Device           Available      Total Type
alpha      /dev/sdb1             2.0T       3.5T  SSD
beta       /dev/sdc1             2.2T       3.5T  SSD
delta      /dev/sde1             2.8T       3.5T  SSD
eneco      /dev/sda1             8.7T        11T  HDD
gamma      /dev/sdd1             3.2T       3.5T  SSD
mantax     /dev/nvme0n1p1        1.1T       5.8T  SSD
# check choice of disk for today
qlv --today
/hd/beta
# start import
qlv -ir
✅:Created directory /hd/beta/qlever/wikidata_20241026
✅:Started screen session qlever_wikidata_20241026.
✅:Logging to /hd/beta/qlever/wikidata_20241026/screen.log
head -1  /hd/beta/qlever/wikidata_20241026/screen.log
Starting QLever indexing process at Sat Oct 26 06:44:55 AM CEST 2024

tail -f  /hd/beta/qlever/wikidata_20241026/screen.log
eval "$(register-python-argcomplete qlever)" && export QLEVER_ARGCOMPLETE_ENABLED=1


Command: get-data

curl -LRC - --remote-name-all https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2 https://dumps.wikimedia.org/wikidatawiki/entities/latest-lexemes.ttl.bz2 2>&1

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0  105G    0  255M    0     0  3859k      0  7:59:43  0:01:07  7:58:36 2054k