WikiData Import 2022-01-29

From BITPlan Wiki
Jump to navigation Jump to search

QLever trial

see https://github.com/ad-freiburg/qlever/blob/master/docs/quickstart.md

see QLever/script as discussed in QLever Issue #562 for the script which makes reproducing this attempt easier.

Environment

Operating System

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

docker version

docker --version
Docker version 19.03.13, build 4484c46d9d

Memory

wf@merkur:~$ free -h
              total        used        free      shared  buff/cache   available
Mem:           62Gi       1,3Gi        60Gi        30Mi       1,4Gi        60Gi
Swap:          18Gi          0B        18Gi

diskspace

 vendor               | model              | size      | days  | years |
----------------------+--------------------+-----------+-------+-------+
  ST6000VX0023-2EF110 |                    |    6,00TB |   300 |   0.8 |
              SanDisk |     SSD PLUS 120GB |     120GB |   237 |   0.6 |
                  WDC |   WD20EARS-00MVWB1 |    2,00TB |  2232 |   6.1 |
   ST8000DM004-2U9188 |                    |    8,00TB |     8 |   0.0 |

Model Family:     Seagate Barracuda Compute
Device Model:     ST8000DM004-2U9188
Firmware Version: 0001
User Capacity:    8.001.563.222.016 bytes [8,00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      3.5 inches

Steps

The steps below maybe automated with the QLever/script

git clone (1 min)

export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Sa 29. Jan 17:12:45 CET 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 12917, done.
remote: Counting objects: 100% (622/622), done.
remote: Compressing objects: 100% (514/514), done.
remote: Total 12917 (delta 382), reused 206 (delta 103), pack-reused 12295
Receiving objects: 100% (12917/12917), 111.10 MiB | 6.81 MiB/s, done.
Resolving deltas: 100% (10067/10067), done.
Submodule 'third_party/abseil-cpp' (https://github.com/abseil/abseil-cpp.git) registered for path 'third_party/abseil-cpp'
Submodule 'third_party/antlr4' (https://github.com/antlr/antlr4.git) registered for path 'third_party/antlr4'
Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest'
Submodule 'third_party/re2' (https://github.com/google/re2.git) registered for path 'third_party/re2'
Submodule 'third_party/stxxl' (https://github.com/ad-freiburg/stxxl) registered for path 'third_party/stxxl'
Cloning into '/hd/jurob/qlever/qlever-code/third_party/abseil-cpp'...
remote: Enumerating objects: 16122, done.        
remote: Counting objects: 100% (248/248), done.        
remote: Compressing objects: 100% (193/193), done.        
remote: Total 16122 (delta 116), reused 106 (delta 55), pack-reused 15874        
Receiving objects: 100% (16122/16122), 10.39 MiB | 6.85 MiB/s, done.
Resolving deltas: 100% (12398/12398), done.
Cloning into '/hd/jurob/qlever/qlever-code/third_party/antlr4'...
remote: Enumerating objects: 125326, done.        
remote: Counting objects: 100% (2734/2734), done.        
remote: Compressing objects: 100% (1022/1022), done.        
remote: Total 125326 (delta 1488), reused 2292 (delta 1246), pack-reused 122592        
Receiving objects: 100% (125326/125326), 64.11 MiB | 6.72 MiB/s, done.
Resolving deltas: 100% (73647/73647), done.
Cloning into '/hd/jurob/qlever/qlever-code/third_party/googletest'...
remote: Enumerating objects: 23795, done.        
remote: Counting objects: 100% (259/259), done.        
remote: Compressing objects: 100% (146/146), done.        
remote: Total 23795 (delta 134), reused 200 (delta 105), pack-reused 23536        
Receiving objects: 100% (23795/23795), 9.90 MiB | 6.89 MiB/s, done.
Resolving deltas: 100% (17517/17517), done.
Cloning into '/hd/jurob/qlever/qlever-code/third_party/re2'...
remote: Enumerating objects: 6996, done.        
remote: Counting objects: 100% (617/617), done.        
remote: Compressing objects: 100% (388/388), done.        
remote: Total 6996 (delta 392), reused 399 (delta 223), pack-reused 6379        
Receiving objects: 100% (6996/6996), 3.49 MiB | 6.94 MiB/s, done.
Resolving deltas: 100% (5287/5287), done.
Cloning into '/hd/jurob/qlever/qlever-code/third_party/stxxl'...
remote: Enumerating objects: 40982, done.        
remote: Counting objects: 100% (45/45), done.        
remote: Compressing objects: 100% (31/31), done.        
remote: Total 40982 (delta 15), reused 28 (delta 7), pack-reused 40937        
Receiving objects: 100% (40982/40982), 14.13 MiB | 6.77 MiB/s, done.
Resolving deltas: 100% (30914/30914), done.
Submodule path 'third_party/abseil-cpp': checked out 'b9b925341f9e90f5e7aa0cf23f036c29c7e454eb'
Submodule path 'third_party/antlr4': checked out 'e4c1a74c66bd5290364ea2b36c97cd724b247357'
Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
Submodule path 'third_party/re2': checked out '13ebb377c6ad763ca61d12dd6f88b1126bd0b911'
Submodule path 'third_party/stxxl': checked out 'a4f884f2a2b4ea078c34c48e1e9a0003f4619f00'
Submodule 'extlib/foxxll' (https://github.com/ad-freiburg/foxxll.git) registered for path 'third_party/stxxl/extlib/foxxll'
Cloning into '/hd/jurob/qlever/qlever-code/third_party/stxxl/extlib/foxxll'...
remote: Enumerating objects: 21414, done.        
remote: Counting objects: 100% (28/28), done.        
remote: Compressing objects: 100% (22/22), done.        
remote: Total 21414 (delta 9), reused 13 (delta 4), pack-reused 21386        
Receiving objects: 100% (21414/21414), 4.60 MiB | 2.48 MiB/s, done.
Resolving deltas: 100% (15789/15789), done.
Submodule path 'third_party/stxxl/extlib/foxxll': checked out '8cbca7bedcdb0b84a6de99e927c5fa27a4bbbfb2'
Submodule 'extlib/tlx' (https://github.com/joka921/tlx.git) registered for path 'third_party/stxxl/extlib/foxxll/extlib/tlx'
Cloning into '/hd/jurob/qlever/qlever-code/third_party/stxxl/extlib/foxxll/extlib/tlx'...
remote: Enumerating objects: 3418, done.        
remote: Counting objects: 100% (53/53), done.        
remote: Compressing objects: 100% (44/44), done.        
remote: Total 3418 (delta 24), reused 22 (delta 9), pack-reused 3365        
Receiving objects: 100% (3418/3418), 1.11 MiB | 2.75 MiB/s, done.
Resolving deltas: 100% (2611/2611), done.
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Sa 29. Jan 17:13:49 CET 2022

docker build (15 mins)

cd qlever-code/
wf@merkur:/hd/jurob/qlever/qlever-code$ date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Sa 29. Jan 17:14:51 CET 2022
Sending build context to Docker daemon  126.1MB
Step 1/43 : FROM ubuntu:20.04 as base
...
Successfully tagged qlever:latest
Sa 29. Jan 17:29:27 CET 2022

Wikidata dump Download (6h)

This had been done on another machine a few days earlier.

date;wget https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2;date
Thu Jan 27 11:08:10 CET 2022
--2022-01-27 11:08:10--  https://dumps.wikimedia.org/wikidatawiki/entities/latest-all.ttl.bz2
Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 2620::861:1:208:80:154:7, 208.80.154.7
Connecting to dumps.wikimedia.org (dumps.wikimedia.org)|2620::861:1:208:80:154:7|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 92422961847 (86G) [application/octet-stream]
Saving to: ‘latest-all.ttl.bz2’

latest-all.ttl.bz2  100%[===================>]  86.08G  4.60MB/s    in 5h 46m  

2022-01-27 16:55:06 (4.23 MB/s) - ‘latest-all.ttl.bz2’ saved [92422961847/92422961847]

Thu Jan 27 16:55:06 CET 2022
date;wget https://dumps.wikimedia.org/wikidatawiki/entities/latest-lexemes.ttl.bz2;date
Thu Jan 27 17:36:38 CET 2022
--2022-01-27 17:36:38--  https://dumps.wikimedia.org/wikidatawiki/entities/latest-lexemes.ttl.bz2
Resolving dumps.wikimedia.org (dumps.wikimedia.org)... 2620::861:1:208:80:154:7, 208.80.154.7
Connecting to dumps.wikimedia.org (dumps.wikimedia.org)|2620::861:1:208:80:154:7|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 315591211 (301M) [application/octet-stream]
Saving to: ‘latest-lexemes.ttl.bz2’

latest-lexemes.ttl. 100%[===================>] 300.97M  4.95MB/s    in 60s     

2022-01-27 17:37:39 (5.00 MB/s) - ‘latest-lexemes.ttl.bz2’ saved [315591211/315591211]

Thu Jan 27 17:37:39 CET 2022

Indexing 4.1 days

./qlever --wikidata_index
2022-02-02 05:53:42.391	- INFO:  �[1mQLever IndexBuilder, compiled on Jan 29 2022 20:19:12�[22m
2022-02-02 05:53:42.418	- INFO:  You specified the input format: TTL
2022-02-02 05:53:42.566	- INFO:  You specified "locale = en_US" and "ignore-punctuation = 1"
2022-02-02 05:53:42.568	- INFO:  You specified "ascii-prefixes-only = true", which enables faster parsing for well-behaved TTL files (see qlever/docs on GitHub)
2022-02-02 05:53:42.568	- INFO:  You specified "num-triples-per-partial-vocab = 20,000,000", choose a lower value if the index builder runs out of memory
2022-02-02 05:53:42.569	- INFO:  Processing input triples from /dev/stdin ...
2022-02-02 05:59:07.200	- INFO:  Input triples processed: 100,000,000
2022-02-02 06:04:25.068	- INFO:  Input triples processed: 200,000,000
2022-02-02 06:08:49.023	- INFO:  Input triples processed: 300,000,000
2022-02-02 06:13:42.180	- INFO:  Input triples processed: 400,000,000
2022-02-02 06:18:35.785	- INFO:  Input triples processed: 500,000,000
2022-02-02 06:23:33.098	- INFO:  Input triples processed: 600,000,000
2022-02-02 06:28:40.453	- INFO:  Input triples processed: 700,000,000
2022-02-02 06:33:33.224	- INFO:  Input triples processed: 800,000,000
2022-02-02 06:38:29.646	- INFO:  Input triples processed: 900,000,000
2022-02-02 06:43:31.615	- INFO:  Input triples processed: 1,000,000,000
2022-02-02 06:48:50.864	- INFO:  Input triples processed: 1,100,000,000
2022-02-02 06:54:44.529	- INFO:  Input triples processed: 1,200,000,000
2022-02-02 07:01:03.669	- INFO:  Input triples processed: 1,300,000,000
2022-02-02 07:06:57.283	- INFO:  Input triples processed: 1,400,000,000
2022-02-02 07:13:02.002	- INFO:  Input triples processed: 1,500,000,000
2022-02-02 07:19:02.355	- INFO:  Input triples processed: 1,600,000,000
2022-02-02 07:25:32.057	- INFO:  Input triples processed: 1,700,000,000
2022-02-02 07:31:41.109	- INFO:  Input triples processed: 1,800,000,000
2022-02-02 07:37:39.025	- INFO:  Input triples processed: 1,900,000,000
2022-02-02 07:43:24.237	- INFO:  Input triples processed: 2,000,000,000
2022-02-02 07:48:33.112	- INFO:  Input triples processed: 2,100,000,000
2022-02-02 07:54:15.348	- INFO:  Input triples processed: 2,200,000,000
2022-02-02 07:59:55.743	- INFO:  Input triples processed: 2,300,000,000
2022-02-02 08:05:09.401	- INFO:  Input triples processed: 2,400,000,000
2022-02-02 08:10:38.497	- INFO:  Input triples processed: 2,500,000,000
2022-02-02 08:16:27.476	- INFO:  Input triples processed: 2,600,000,000
2022-02-02 08:21:52.338	- INFO:  Input triples processed: 2,700,000,000
2022-02-02 08:27:46.142	- INFO:  Input triples processed: 2,800,000,000
2022-02-02 08:33:28.374	- INFO:  Input triples processed: 2,900,000,000
2022-02-02 08:39:30.528	- INFO:  Input triples processed: 3,000,000,000
2022-02-02 08:45:08.281	- INFO:  Input triples processed: 3,100,000,000
2022-02-02 08:50:52.437	- INFO:  Input triples processed: 3,200,000,000
2022-02-02 09:03:13.800	- INFO:  Input triples processed: 3,300,000,000
2022-02-02 09:08:56.368	- INFO:  Input triples processed: 3,400,000,000
2022-02-02 09:14:37.689	- INFO:  Input triples processed: 3,500,000,000
2022-02-02 09:20:14.816	- INFO:  Input triples processed: 3,600,000,000
2022-02-02 09:25:46.666	- INFO:  Input triples processed: 3,700,000,000
2022-02-02 09:31:28.112	- INFO:  Input triples processed: 3,800,000,000
2022-02-02 09:36:55.886	- INFO:  Input triples processed: 3,900,000,000
2022-02-02 09:42:28.592	- INFO:  Input triples processed: 4,000,000,000
2022-02-02 09:48:02.848	- INFO:  Input triples processed: 4,100,000,000
2022-02-02 09:53:16.606	- INFO:  Input triples processed: 4,200,000,000
2022-02-02 09:58:40.663	- INFO:  Input triples processed: 4,300,000,000
2022-02-02 10:04:17.846	- INFO:  Input triples processed: 4,400,000,000
2022-02-02 10:09:21.045	- INFO:  Input triples processed: 4,500,000,000
2022-02-02 10:14:34.149	- INFO:  Input triples processed: 4,600,000,000
2022-02-02 10:19:51.519	- INFO:  Input triples processed: 4,700,000,000
2022-02-02 10:25:54.075	- INFO:  Input triples processed: 4,800,000,000
2022-02-02 10:31:12.738	- INFO:  Input triples processed: 4,900,000,000
2022-02-02 10:36:28.453	- INFO:  Input triples processed: 5,000,000,000
2022-02-02 10:41:58.489	- INFO:  Input triples processed: 5,100,000,000
2022-02-02 10:47:19.305	- INFO:  Input triples processed: 5,200,000,000
2022-02-02 10:53:03.096	- INFO:  Input triples processed: 5,300,000,000
2022-02-02 10:58:30.772	- INFO:  Input triples processed: 5,400,000,000
2022-02-02 11:03:53.039	- INFO:  Input triples processed: 5,500,000,000
2022-02-02 11:09:07.835	- INFO:  Input triples processed: 5,600,000,000
2022-02-02 11:14:43.148	- INFO:  Input triples processed: 5,700,000,000
2022-02-02 11:20:30.355	- INFO:  Input triples processed: 5,800,000,000
2022-02-02 11:26:12.168	- INFO:  Input triples processed: 5,900,000,000
2022-02-02 11:31:49.249	- INFO:  Input triples processed: 6,000,000,000
2022-02-02 11:37:42.195	- INFO:  Input triples processed: 6,100,000,000
2022-02-02 11:43:14.152	- INFO:  Input triples processed: 6,200,000,000
2022-02-02 11:48:56.857	- INFO:  Input triples processed: 6,300,000,000
2022-02-02 11:54:14.783	- INFO:  Input triples processed: 6,400,000,000
2022-02-02 11:59:44.683	- INFO:  Input triples processed: 6,500,000,000
2022-02-02 12:04:57.135	- INFO:  Input triples processed: 6,600,000,000
2022-02-02 12:10:12.283	- INFO:  Input triples processed: 6,700,000,000
2022-02-02 12:15:35.776	- INFO:  Input triples processed: 6,800,000,000
2022-02-02 12:20:38.895	- INFO:  Input triples processed: 6,900,000,000
2022-02-02 12:26:11.903	- INFO:  Input triples processed: 7,000,000,000
2022-02-02 12:31:24.482	- INFO:  Input triples processed: 7,100,000,000
2022-02-02 12:36:43.304	- INFO:  Input triples processed: 7,200,000,000
2022-02-02 12:42:26.933	- INFO:  Input triples processed: 7,300,000,000
2022-02-02 12:47:32.521	- INFO:  Input triples processed: 7,400,000,000
2022-02-02 12:53:06.737	- INFO:  Input triples processed: 7,500,000,000
2022-02-02 12:58:50.177	- INFO:  Input triples processed: 7,600,000,000
2022-02-02 13:04:13.764	- INFO:  Input triples processed: 7,700,000,000
2022-02-02 13:09:53.474	- INFO:  Input triples processed: 7,800,000,000
2022-02-02 13:15:30.449	- INFO:  Input triples processed: 7,900,000,000
2022-02-02 13:21:18.666	- INFO:  Input triples processed: 8,000,000,000
2022-02-02 13:27:02.409	- INFO:  Input triples processed: 8,100,000,000
2022-02-02 13:32:37.830	- INFO:  Input triples processed: 8,200,000,000
2022-02-02 13:38:08.812	- INFO:  Input triples processed: 8,300,000,000
2022-02-02 13:43:19.219	- INFO:  Input triples processed: 8,400,000,000
2022-02-02 13:48:43.536	- INFO:  Input triples processed: 8,500,000,000
2022-02-02 13:54:49.380	- INFO:  Input triples processed: 8,600,000,000
2022-02-02 14:00:03.529	- INFO:  Input triples processed: 8,700,000,000
2022-02-02 14:05:07.486	- INFO:  Input triples processed: 8,800,000,000
2022-02-02 14:10:24.776	- INFO:  Input triples processed: 8,900,000,000
2022-02-02 14:16:03.699	- INFO:  Input triples processed: 9,000,000,000
2022-02-02 14:21:17.048	- INFO:  Input triples processed: 9,100,000,000
2022-02-02 14:26:38.329	- INFO:  Input triples processed: 9,200,000,000
2022-02-02 14:32:07.014	- INFO:  Input triples processed: 9,300,000,000
2022-02-02 14:37:35.128	- INFO:  Input triples processed: 9,400,000,000
2022-02-02 14:42:42.646	- INFO:  Input triples processed: 9,500,000,000
2022-02-02 14:47:55.735	- INFO:  Input triples processed: 9,600,000,000
2022-02-02 14:53:25.586	- INFO:  Input triples processed: 9,700,000,000
2022-02-02 14:58:45.841	- INFO:  Input triples processed: 9,800,000,000
2022-02-02 15:04:14.244	- INFO:  Input triples processed: 9,900,000,000
2022-02-02 15:09:45.942	- INFO:  Input triples processed: 10,000,000,000
2022-02-02 15:15:29.977	- INFO:  Input triples processed: 10,100,000,000
2022-02-02 15:21:02.689	- INFO:  Input triples processed: 10,200,000,000
2022-02-02 15:26:50.306	- INFO:  Input triples processed: 10,300,000,000
2022-02-02 15:32:30.487	- INFO:  Input triples processed: 10,400,000,000
2022-02-02 15:37:55.973	- INFO:  Input triples processed: 10,500,000,000
2022-02-02 15:43:21.336	- INFO:  Input triples processed: 10,600,000,000
2022-02-02 15:49:07.198	- INFO:  Input triples processed: 10,700,000,000
2022-02-02 15:54:21.597	- INFO:  Input triples processed: 10,800,000,000
2022-02-02 15:59:26.248	- INFO:  Input triples processed: 10,900,000,000
2022-02-02 16:04:47.254	- INFO:  Input triples processed: 11,000,000,000
2022-02-02 16:10:21.939	- INFO:  Input triples processed: 11,100,000,000
2022-02-02 16:16:17.855	- INFO:  Input triples processed: 11,200,000,000
2022-02-02 16:21:46.377	- INFO:  Input triples processed: 11,300,000,000
2022-02-02 16:27:05.461	- INFO:  Input triples processed: 11,400,000,000
2022-02-02 16:32:31.003	- INFO:  Input triples processed: 11,500,000,000
2022-02-02 16:37:43.081	- INFO:  Input triples processed: 11,600,000,000
2022-02-02 16:43:14.589	- INFO:  Input triples processed: 11,700,000,000
2022-02-02 16:48:55.122	- INFO:  Input triples processed: 11,800,000,000
2022-02-02 16:54:17.942	- INFO:  Input triples processed: 11,900,000,000
2022-02-02 16:59:48.271	- INFO:  Input triples processed: 12,000,000,000
2022-02-02 17:05:31.450	- INFO:  Input triples processed: 12,100,000,000
2022-02-02 17:11:21.730	- INFO:  Input triples processed: 12,200,000,000
2022-02-02 17:17:07.859	- INFO:  Input triples processed: 12,300,000,000
2022-02-02 17:22:38.664	- INFO:  Input triples processed: 12,400,000,000
2022-02-02 17:28:27.188	- INFO:  Input triples processed: 12,500,000,000
2022-02-02 17:33:53.612	- INFO:  Input triples processed: 12,600,000,000
2022-02-02 17:39:21.492	- INFO:  Input triples processed: 12,700,000,000
2022-02-02 17:45:10.469	- INFO:  Input triples processed: 12,800,000,000
2022-02-02 17:50:24.948	- INFO:  Input triples processed: 12,900,000,000
2022-02-02 17:55:29.110	- INFO:  Input triples processed: 13,000,000,000
2022-02-02 18:00:57.257	- INFO:  Input triples processed: 13,100,000,000
2022-02-02 18:06:17.378	- INFO:  Input triples processed: 13,200,000,000
2022-02-02 18:11:47.981	- INFO:  Input triples processed: 13,300,000,000
2022-02-02 18:17:10.888	- INFO:  Input triples processed: 13,400,000,000
2022-02-02 18:22:47.533	- INFO:  Input triples processed: 13,500,000,000
2022-02-02 18:28:01.125	- INFO:  Input triples processed: 13,600,000,000
2022-02-02 18:33:20.535	- INFO:  Input triples processed: 13,700,000,000
2022-02-02 18:39:01.344	- INFO:  Input triples processed: 13,800,000,000
2022-02-02 18:44:15.241	- INFO:  Input triples processed: 13,900,000,000
2022-02-02 18:50:00.600	- INFO:  Input triples processed: 14,000,000,000
2022-02-02 18:55:33.580	- INFO:  Input triples processed: 14,100,000,000
2022-02-02 19:01:14.666	- INFO:  Input triples processed: 14,200,000,000
2022-02-02 19:06:46.602	- INFO:  Input triples processed: 14,300,000,000
2022-02-02 19:12:38.371	- INFO:  Input triples processed: 14,400,000,000
2022-02-02 19:18:41.261	- INFO:  Input triples processed: 14,500,000,000
2022-02-02 19:24:17.925	- INFO:  Input triples processed: 14,600,000,000
2022-02-02 19:29:31.631	- INFO:  Input triples processed: 14,700,000,000
2022-02-02 19:35:08.007	- INFO:  Input triples processed: 14,800,000,000
2022-02-02 19:40:52.112	- INFO:  Input triples processed: 14,900,000,000
2022-02-02 19:46:00.751	- INFO:  Input triples processed: 15,000,000,000
2022-02-02 19:51:10.905	- INFO:  Input triples processed: 15,100,000,000
2022-02-02 19:56:46.037	- INFO:  Input triples processed: 15,200,000,000
2022-02-02 20:02:09.632	- INFO:  Input triples processed: 15,300,000,000
2022-02-02 20:07:34.112	- INFO:  Input triples processed: 15,400,000,000
2022-02-02 20:12:41.231	- INFO:  Input triples processed: 15,500,000,000
2022-02-02 20:18:17.512	- INFO:  Input triples processed: 15,600,000,000
2022-02-02 20:23:50.285	- INFO:  Input triples processed: 15,700,000,000
2022-02-02 20:29:25.067	- INFO:  Input triples processed: 15,800,000,000
2022-02-02 20:34:45.926	- INFO:  Input triples processed: 15,900,000,000
2022-02-02 20:40:08.984	- INFO:  Input triples processed: 16,000,000,000
2022-02-02 20:45:35.751	- INFO:  Input triples processed: 16,100,000,000
2022-02-02 20:51:10.034	- INFO:  Input triples processed: 16,200,000,000
2022-02-02 20:57:02.400	- INFO:  Input triples processed: 16,300,000,000
2022-02-02 21:02:45.497	- INFO:  Input triples processed: 16,400,000,000
2022-02-02 21:08:34.531	- INFO:  Input triples processed: 16,500,000,000
2022-02-02 21:14:14.133	- INFO:  Input triples processed: 16,600,000,000
2022-02-02 21:19:58.155	- INFO:  Input triples processed: 16,700,000,000
2022-02-02 21:25:23.936	- INFO:  Input triples processed: 16,800,000,000
2022-02-02 21:30:23.323	- INFO:  Input triples processed: 16,900,000,000
2022-02-02 21:35:25.492	- INFO:  Done, total number of triples read: 16,979,598,468 [may contain duplicates]
2022-02-02 21:35:25.492	- INFO:  Number of QLever-internal triples created: 10,093,281,840 [may contain duplicates]
2022-02-02 21:35:25.492	- INFO:  Merging partial vocabularies in byte order (internal only) ...
2022-02-02 21:40:46.606	- INFO:  Words merged: 100,000,000
2022-02-02 21:48:22.776	- INFO:  Words merged: 200,000,000
2022-02-02 21:55:53.025	- INFO:  Words merged: 300,000,000
2022-02-02 22:06:50.288	- INFO:  Words merged: 400,000,000
2022-02-02 22:15:20.636	- INFO:  Words merged: 500,000,000
2022-02-02 22:21:49.518	- INFO:  Words merged: 600,000,000
2022-02-02 22:43:44.014	- INFO:  Words merged: 700,000,000
2022-02-02 22:50:53.562	- INFO:  Words merged: 800,000,000
2022-02-02 22:56:18.623	- INFO:  Words merged: 900,000,000
2022-02-02 22:57:48.601	- INFO:  Number of words in internal vocabulary: 938,491,990
2022-02-02 22:57:48.602	- INFO:  Merging partial vocabularies in Unicode order (internal and external) ...
2022-02-02 23:06:56.472	- INFO:  Words merged: 100,000,000
2022-02-02 23:17:33.337	- INFO:  Words merged: 200,000,000
2022-02-02 23:28:18.214	- INFO:  Words merged: 300,000,000
2022-02-02 23:42:45.536	- INFO:  Words merged: 400,000,000
2022-02-02 23:52:58.541	- INFO:  Words merged: 500,000,000
2022-02-03 00:01:16.765	- INFO:  Words merged: 600,000,000
2022-02-03 00:11:23.418	- INFO:  Words merged: 700,000,000
2022-02-03 00:19:46.619	- INFO:  Words merged: 800,000,000
2022-02-03 00:45:36.524	- INFO:  Words merged: 900,000,000
2022-02-03 00:58:09.284	- INFO:  Words merged: 1,000,000,000
2022-02-03 01:06:24.768	- INFO:  Words merged: 1,100,000,000
2022-02-03 01:14:50.087	- INFO:  Words merged: 1,200,000,000
2022-02-03 01:23:12.324	- INFO:  Words merged: 1,300,000,000
2022-02-03 01:33:05.021	- INFO:  Words merged: 1,400,000,000
2022-02-03 01:41:01.953	- INFO:  Words merged: 1,500,000,000
2022-02-03 01:50:09.204	- INFO:  Words merged: 1,600,000,000
2022-02-03 02:02:03.331	- INFO:  Words merged: 1,700,000,000
2022-02-03 02:13:53.724	- INFO:  Words merged: 1,800,000,000
2022-02-03 02:22:53.444	- INFO:  Words merged: 1,900,000,000
2022-02-03 02:32:32.792	- INFO:  Words merged: 2,000,000,000
2022-02-03 02:42:29.268	- INFO:  Words merged: 2,100,000,000
2022-02-03 02:52:16.904	- INFO:  Words merged: 2,200,000,000
2022-02-03 03:02:03.858	- INFO:  Words merged: 2,300,000,000
2022-02-03 03:10:43.495	- INFO:  Words merged: 2,400,000,000
2022-02-03 03:20:05.744	- INFO:  Words merged: 2,500,000,000
2022-02-03 03:29:44.044	- INFO:  Words merged: 2,600,000,000
2022-02-03 03:38:34.892	- INFO:  Words merged: 2,700,000,000
2022-02-03 03:47:25.028	- INFO:  Words merged: 2,800,000,000
2022-02-03 03:56:30.070	- INFO:  Words merged: 2,900,000,000
2022-02-03 04:05:56.032	- INFO:  Words merged: 3,000,000,000
2022-02-03 04:16:10.871	- INFO:  Words merged: 3,100,000,000
2022-02-03 04:30:33.889	- INFO:  Words merged: 3,200,000,000
2022-02-03 04:38:45.525	- INFO:  Number of words in external vocabulary: 2,289,883,113
2022-02-03 04:38:45.531	- INFO:  Removing temporary files ...
2022-02-03 04:44:04.818	- INFO:  Converting external vocabulary to binary format ...
2022-02-03 06:04:44.060	- INFO:  Converting triples from local IDs to global IDs ...
2022-02-03 06:07:03.695	- INFO:  Triples converted: 100,000,000
2022-02-03 06:08:47.794	- INFO:  Triples converted: 200,000,000
2022-02-03 06:10:29.184	- INFO:  Triples converted: 300,000,000
2022-02-03 06:12:20.313	- INFO:  Triples converted: 400,000,000
2022-02-03 06:14:02.908	- INFO:  Triples converted: 500,000,000
2022-02-03 06:15:36.523	- INFO:  Triples converted: 600,000,000
2022-02-03 06:17:07.086	- INFO:  Triples converted: 700,000,000
2022-02-03 06:18:31.008	- INFO:  Triples converted: 800,000,000
2022-02-03 06:19:50.168	- INFO:  Triples converted: 900,000,000
2022-02-03 06:21:21.558	- INFO:  Triples converted: 1,000,000,000
2022-02-03 06:22:52.771	- INFO:  Triples converted: 1,100,000,000
2022-02-03 06:24:32.112	- INFO:  Triples converted: 1,200,000,000
2022-02-03 06:25:50.747	- INFO:  Triples converted: 1,300,000,000
2022-02-03 06:27:14.209	- INFO:  Triples converted: 1,400,000,000
2022-02-03 06:28:50.732	- INFO:  Triples converted: 1,500,000,000
2022-02-03 06:30:33.606	- INFO:  Triples converted: 1,600,000,000
2022-02-03 06:32:01.607	- INFO:  Triples converted: 1,700,000,000
2022-02-03 06:33:27.405	- INFO:  Triples converted: 1,800,000,000
2022-02-03 06:35:04.990	- INFO:  Triples converted: 1,900,000,000
2022-02-03 06:36:33.487	- INFO:  Triples converted: 2,000,000,000
2022-02-03 06:38:27.729	- INFO:  Triples converted: 2,100,000,000
2022-02-03 06:40:19.789	- INFO:  Triples converted: 2,200,000,000
2022-02-03 06:42:23.455	- INFO:  Triples converted: 2,300,000,000
2022-02-03 06:44:20.406	- INFO:  Triples converted: 2,400,000,000
2022-02-03 06:46:02.518	- INFO:  Triples converted: 2,500,000,000
2022-02-03 06:47:43.823	- INFO:  Triples converted: 2,600,000,000
2022-02-03 06:49:18.014	- INFO:  Triples converted: 2,700,000,000
2022-02-03 06:50:58.714	- INFO:  Triples converted: 2,800,000,000
2022-02-03 06:52:29.128	- INFO:  Triples converted: 2,900,000,000
2022-02-03 06:54:09.454	- INFO:  Triples converted: 3,000,000,000
2022-02-03 06:55:54.293	- INFO:  Triples converted: 3,100,000,000
2022-02-03 06:57:27.644	- INFO:  Triples converted: 3,200,000,000
2022-02-03 06:59:03.790	- INFO:  Triples converted: 3,300,000,000
2022-02-03 07:00:43.280	- INFO:  Triples converted: 3,400,000,000
2022-02-03 07:02:46.205	- INFO:  Triples converted: 3,500,000,000
2022-02-03 07:04:29.008	- INFO:  Triples converted: 3,600,000,000
2022-02-03 07:06:13.559	- INFO:  Triples converted: 3,700,000,000
2022-02-03 07:07:49.482	- INFO:  Triples converted: 3,800,000,000
2022-02-03 07:09:18.854	- INFO:  Triples converted: 3,900,000,000
2022-02-03 07:11:10.058	- INFO:  Triples converted: 4,000,000,000
2022-02-03 07:12:48.747	- INFO:  Triples converted: 4,100,000,000
2022-02-03 07:14:31.783	- INFO:  Triples converted: 4,200,000,000
2022-02-03 07:16:14.597	- INFO:  Triples converted: 4,300,000,000
2022-02-03 07:17:47.847	- INFO:  Triples converted: 4,400,000,000
2022-02-03 07:19:27.982	- INFO:  Triples converted: 4,500,000,000
2022-02-03 07:21:19.428	- INFO:  Triples converted: 4,600,000,000
2022-02-03 07:23:09.734	- INFO:  Triples converted: 4,700,000,000
2022-02-03 07:25:06.433	- INFO:  Triples converted: 4,800,000,000
2022-02-03 07:26:47.995	- INFO:  Triples converted: 4,900,000,000
2022-02-03 07:28:21.895	- INFO:  Triples converted: 5,000,000,000
2022-02-03 07:30:04.774	- INFO:  Triples converted: 5,100,000,000
2022-02-03 07:31:42.133	- INFO:  Triples converted: 5,200,000,000
2022-02-03 07:33:36.029	- INFO:  Triples converted: 5,300,000,000
2022-02-03 07:35:23.311	- INFO:  Triples converted: 5,400,000,000
2022-02-03 07:37:02.714	- INFO:  Triples converted: 5,500,000,000
2022-02-03 07:38:42.429	- INFO:  Triples converted: 5,600,000,000
2022-02-03 07:40:13.634	- INFO:  Triples converted: 5,700,000,000
2022-02-03 07:41:56.442	- INFO:  Triples converted: 5,800,000,000
2022-02-03 07:43:39.667	- INFO:  Triples converted: 5,900,000,000
2022-02-03 07:45:23.077	- INFO:  Triples converted: 6,000,000,000
2022-02-03 07:46:55.523	- INFO:  Triples converted: 6,100,000,000
2022-02-03 07:48:32.532	- INFO:  Triples converted: 6,200,000,000
2022-02-03 07:50:03.494	- INFO:  Triples converted: 6,300,000,000
2022-02-03 07:51:37.785	- INFO:  Triples converted: 6,400,000,000
2022-02-03 07:53:15.261	- INFO:  Triples converted: 6,500,000,000
2022-02-03 07:55:03.364	- INFO:  Triples converted: 6,600,000,000
2022-02-03 07:56:46.808	- INFO:  Triples converted: 6,700,000,000
2022-02-03 07:58:34.540	- INFO:  Triples converted: 6,800,000,000
2022-02-03 08:00:20.230	- INFO:  Triples converted: 6,900,000,000
2022-02-03 08:02:20.105	- INFO:  Triples converted: 7,000,000,000
2022-02-03 08:03:54.073	- INFO:  Triples converted: 7,100,000,000
2022-02-03 08:05:25.878	- INFO:  Triples converted: 7,200,000,000
2022-02-03 08:07:00.337	- INFO:  Triples converted: 7,300,000,000
2022-02-03 08:08:31.856	- INFO:  Triples converted: 7,400,000,000
2022-02-03 08:10:10.808	- INFO:  Triples converted: 7,500,000,000
2022-02-03 08:11:52.730	- INFO:  Triples converted: 7,600,000,000
2022-02-03 08:13:20.423	- INFO:  Triples converted: 7,700,000,000
2022-02-03 08:14:48.426	- INFO:  Triples converted: 7,800,000,000
2022-02-03 08:16:19.342	- INFO:  Triples converted: 7,900,000,000
2022-02-03 08:17:37.066	- INFO:  Triples converted: 8,000,000,000
2022-02-03 08:19:06.312	- INFO:  Triples converted: 8,100,000,000
2022-02-03 08:20:24.640	- INFO:  Triples converted: 8,200,000,000
2022-02-03 08:21:51.644	- INFO:  Triples converted: 8,300,000,000
2022-02-03 08:23:20.594	- INFO:  Triples converted: 8,400,000,000
2022-02-03 08:25:06.193	- INFO:  Triples converted: 8,500,000,000
2022-02-03 08:26:48.557	- INFO:  Triples converted: 8,600,000,000
2022-02-03 08:28:14.023	- INFO:  Triples converted: 8,700,000,000
2022-02-03 08:29:43.745	- INFO:  Triples converted: 8,800,000,000
2022-02-03 08:31:02.245	- INFO:  Triples converted: 8,900,000,000
2022-02-03 08:32:32.198	- INFO:  Triples converted: 9,000,000,000
2022-02-03 08:34:53.199	- INFO:  Triples converted: 9,100,000,000
2022-02-03 08:37:01.970	- INFO:  Triples converted: 9,200,000,000
2022-02-03 08:38:46.399	- INFO:  Triples converted: 9,300,000,000
2022-02-03 08:40:24.404	- INFO:  Triples converted: 9,400,000,000
2022-02-03 08:42:26.291	- INFO:  Triples converted: 9,500,000,000
2022-02-03 08:45:21.894	- INFO:  Triples converted: 9,600,000,000
2022-02-03 08:48:12.170	- INFO:  Triples converted: 9,700,000,000
2022-02-03 08:52:29.881	- INFO:  Triples converted: 9,800,000,000
2022-02-03 08:55:36.916	- INFO:  Triples converted: 9,900,000,000
2022-02-03 08:58:56.555	- INFO:  Triples converted: 10,000,000,000
2022-02-03 09:00:24.102	- INFO:  Triples converted: 10,100,000,000
2022-02-03 09:01:57.289	- INFO:  Triples converted: 10,200,000,000
2022-02-03 09:03:38.529	- INFO:  Triples converted: 10,300,000,000
2022-02-03 09:05:27.127	- INFO:  Triples converted: 10,400,000,000
2022-02-03 09:07:00.262	- INFO:  Triples converted: 10,500,000,000
2022-02-03 09:08:29.934	- INFO:  Triples converted: 10,600,000,000
2022-02-03 09:09:54.909	- INFO:  Triples converted: 10,700,000,000
2022-02-03 09:11:31.481	- INFO:  Triples converted: 10,800,000,000
2022-02-03 09:13:08.048	- INFO:  Triples converted: 10,900,000,000
2022-02-03 09:14:45.819	- INFO:  Triples converted: 11,000,000,000
2022-02-03 09:16:25.319	- INFO:  Triples converted: 11,100,000,000
2022-02-03 09:17:49.967	- INFO:  Triples converted: 11,200,000,000
2022-02-03 09:19:25.744	- INFO:  Triples converted: 11,300,000,000
2022-02-03 09:21:11.002	- INFO:  Triples converted: 11,400,000,000
2022-02-03 09:23:01.497	- INFO:  Triples converted: 11,500,000,000
2022-02-03 09:24:52.943	- INFO:  Triples converted: 11,600,000,000
2022-02-03 09:26:21.260	- INFO:  Triples converted: 11,700,000,000
2022-02-03 09:27:49.169	- INFO:  Triples converted: 11,800,000,000
2022-02-03 09:29:26.027	- INFO:  Triples converted: 11,900,000,000
2022-02-03 09:30:53.202	- INFO:  Triples converted: 12,000,000,000
2022-02-03 09:32:18.027	- INFO:  Triples converted: 12,100,000,000
2022-02-03 09:33:59.259	- INFO:  Triples converted: 12,200,000,000
2022-02-03 09:35:26.412	- INFO:  Triples converted: 12,300,000,000
2022-02-03 09:37:15.019	- INFO:  Triples converted: 12,400,000,000
2022-02-03 09:39:11.756	- INFO:  Triples converted: 12,500,000,000
2022-02-03 09:40:47.331	- INFO:  Triples converted: 12,600,000,000
2022-02-03 09:42:30.756	- INFO:  Triples converted: 12,700,000,000
2022-02-03 09:44:14.363	- INFO:  Triples converted: 12,800,000,000
2022-02-03 09:46:09.220	- INFO:  Triples converted: 12,900,000,000
2022-02-03 09:47:48.549	- INFO:  Triples converted: 13,000,000,000
2022-02-03 09:49:23.978	- INFO:  Triples converted: 13,100,000,000
2022-02-03 09:51:25.475	- INFO:  Triples converted: 13,200,000,000
2022-02-03 09:53:00.696	- INFO:  Triples converted: 13,300,000,000
2022-02-03 09:54:56.191	- INFO:  Triples converted: 13,400,000,000
2022-02-03 09:56:46.508	- INFO:  Triples converted: 13,500,000,000
2022-02-03 09:58:21.403	- INFO:  Triples converted: 13,600,000,000
2022-02-03 10:00:09.504	- INFO:  Triples converted: 13,700,000,000
2022-02-03 10:02:17.575	- INFO:  Triples converted: 13,800,000,000
2022-02-03 10:04:02.820	- INFO:  Triples converted: 13,900,000,000
2022-02-03 10:05:44.401	- INFO:  Triples converted: 14,000,000,000
2022-02-03 10:07:17.813	- INFO:  Triples converted: 14,100,000,000
2022-02-03 10:09:10.136	- INFO:  Triples converted: 14,200,000,000
2022-02-03 10:10:56.520	- INFO:  Triples converted: 14,300,000,000
2022-02-03 10:13:01.286	- INFO:  Triples converted: 14,400,000,000
2022-02-03 10:14:50.223	- INFO:  Triples converted: 14,500,000,000
2022-02-03 10:16:40.323	- INFO:  Triples converted: 14,600,000,000
2022-02-03 10:18:24.466	- INFO:  Triples converted: 14,700,000,000
2022-02-03 10:20:11.680	- INFO:  Triples converted: 14,800,000,000
2022-02-03 10:22:01.493	- INFO:  Triples converted: 14,900,000,000
2022-02-03 10:24:08.333	- INFO:  Triples converted: 15,000,000,000
2022-02-03 10:25:41.022	- INFO:  Triples converted: 15,100,000,000
2022-02-03 10:27:25.221	- INFO:  Triples converted: 15,200,000,000
2022-02-03 10:29:00.946	- INFO:  Triples converted: 15,300,000,000
2022-02-03 10:30:52.184	- INFO:  Triples converted: 15,400,000,000
2022-02-03 10:32:38.721	- INFO:  Triples converted: 15,500,000,000
2022-02-03 10:34:11.556	- INFO:  Triples converted: 15,600,000,000
2022-02-03 10:35:57.410	- INFO:  Triples converted: 15,700,000,000
2022-02-03 10:37:44.445	- INFO:  Triples converted: 15,800,000,000
2022-02-03 10:39:16.599	- INFO:  Triples converted: 15,900,000,000
2022-02-03 10:41:02.244	- INFO:  Triples converted: 16,000,000,000
2022-02-03 10:42:39.534	- INFO:  Triples converted: 16,100,000,000
2022-02-03 10:44:19.688	- INFO:  Triples converted: 16,200,000,000
2022-02-03 10:46:13.325	- INFO:  Triples converted: 16,300,000,000
2022-02-03 10:47:55.063	- INFO:  Triples converted: 16,400,000,000
2022-02-03 10:49:42.592	- INFO:  Triples converted: 16,500,000,000
2022-02-03 10:51:27.606	- INFO:  Triples converted: 16,600,000,000
2022-02-03 10:53:25.996	- INFO:  Triples converted: 16,700,000,000
2022-02-03 10:55:20.118	- INFO:  Triples converted: 16,800,000,000
2022-02-03 10:57:21.513	- INFO:  Triples converted: 16,900,000,000
2022-02-03 10:59:16.479	- INFO:  Triples converted: 17,000,000,000
2022-02-03 11:01:11.333	- INFO:  Triples converted: 17,100,000,000
2022-02-03 11:03:04.520	- INFO:  Triples converted: 17,200,000,000
2022-02-03 11:04:40.493	- INFO:  Triples converted: 17,300,000,000
2022-02-03 11:06:28.389	- INFO:  Triples converted: 17,400,000,000
2022-02-03 11:08:26.442	- INFO:  Triples converted: 17,500,000,000
2022-02-03 11:10:22.786	- INFO:  Triples converted: 17,600,000,000
2022-02-03 11:12:36.616	- INFO:  Triples converted: 17,700,000,000
2022-02-03 11:14:18.778	- INFO:  Triples converted: 17,800,000,000
2022-02-03 11:16:13.166	- INFO:  Triples converted: 17,900,000,000
2022-02-03 11:18:10.780	- INFO:  Triples converted: 18,000,000,000
2022-02-03 11:19:51.236	- INFO:  Triples converted: 18,100,000,000
2022-02-03 11:21:40.897	- INFO:  Triples converted: 18,200,000,000
2022-02-03 11:23:21.193	- INFO:  Triples converted: 18,300,000,000
2022-02-03 11:25:03.483	- INFO:  Triples converted: 18,400,000,000
2022-02-03 11:26:57.531	- INFO:  Triples converted: 18,500,000,000
2022-02-03 11:28:27.511	- INFO:  Triples converted: 18,600,000,000
2022-02-03 11:30:11.088	- INFO:  Triples converted: 18,700,000,000
2022-02-03 11:31:54.527	- INFO:  Triples converted: 18,800,000,000
2022-02-03 11:33:36.040	- INFO:  Triples converted: 18,900,000,000
2022-02-03 11:35:17.586	- INFO:  Triples converted: 19,000,000,000
2022-02-03 11:37:06.543	- INFO:  Triples converted: 19,100,000,000
2022-02-03 11:38:49.740	- INFO:  Triples converted: 19,200,000,000
2022-02-03 11:40:42.584	- INFO:  Triples converted: 19,300,000,000
2022-02-03 11:42:30.329	- INFO:  Triples converted: 19,400,000,000
2022-02-03 11:44:19.331	- INFO:  Triples converted: 19,500,000,000
2022-02-03 11:46:00.269	- INFO:  Triples converted: 19,600,000,000
2022-02-03 11:47:37.169	- INFO:  Triples converted: 19,700,000,000
2022-02-03 11:49:38.635	- INFO:  Triples converted: 19,800,000,000
2022-02-03 11:51:22.625	- INFO:  Triples converted: 19,900,000,000
2022-02-03 11:53:26.662	- INFO:  Triples converted: 20,000,000,000
2022-02-03 11:55:20.818	- INFO:  Triples converted: 20,100,000,000
2022-02-03 11:56:57.081	- INFO:  Triples converted: 20,200,000,000
2022-02-03 11:58:44.907	- INFO:  Triples converted: 20,300,000,000
2022-02-03 12:00:41.842	- INFO:  Triples converted: 20,400,000,000
2022-02-03 12:02:39.025	- INFO:  Triples converted: 20,500,000,000
2022-02-03 12:04:19.605	- INFO:  Triples converted: 20,600,000,000
2022-02-03 12:06:05.027	- INFO:  Triples converted: 20,700,000,000
2022-02-03 12:07:47.944	- INFO:  Triples converted: 20,800,000,000
2022-02-03 12:09:42.388	- INFO:  Triples converted: 20,900,000,000
2022-02-03 12:11:38.584	- INFO:  Triples converted: 21,000,000,000
2022-02-03 12:13:28.775	- INFO:  Triples converted: 21,100,000,000
2022-02-03 12:15:22.070	- INFO:  Triples converted: 21,200,000,000
2022-02-03 12:17:03.204	- INFO:  Triples converted: 21,300,000,000
2022-02-03 12:18:44.107	- INFO:  Triples converted: 21,400,000,000
2022-02-03 12:20:20.415	- INFO:  Triples converted: 21,500,000,000
2022-02-03 12:22:13.964	- INFO:  Triples converted: 21,600,000,000
2022-02-03 12:24:18.256	- INFO:  Triples converted: 21,700,000,000
2022-02-03 12:26:08.881	- INFO:  Triples converted: 21,800,000,000
2022-02-03 12:27:48.096	- INFO:  Triples converted: 21,900,000,000
2022-02-03 12:29:18.359	- INFO:  Triples converted: 22,000,000,000
2022-02-03 12:31:01.268	- INFO:  Triples converted: 22,100,000,000
2022-02-03 12:32:45.265	- INFO:  Triples converted: 22,200,000,000
2022-02-03 12:34:28.333	- INFO:  Triples converted: 22,300,000,000
2022-02-03 12:36:18.373	- INFO:  Triples converted: 22,400,000,000
2022-02-03 12:38:12.105	- INFO:  Triples converted: 22,500,000,000
2022-02-03 12:40:03.848	- INFO:  Triples converted: 22,600,000,000
2022-02-03 12:41:37.959	- INFO:  Triples converted: 22,700,000,000
2022-02-03 12:43:24.983	- INFO:  Triples converted: 22,800,000,000
2022-02-03 12:45:20.230	- INFO:  Triples converted: 22,900,000,000
2022-02-03 12:47:20.548	- INFO:  Triples converted: 23,000,000,000
2022-02-03 12:49:18.133	- INFO:  Triples converted: 23,100,000,000
2022-02-03 12:51:04.519	- INFO:  Triples converted: 23,200,000,000
2022-02-03 12:52:58.685	- INFO:  Triples converted: 23,300,000,000
2022-02-03 12:54:54.395	- INFO:  Triples converted: 23,400,000,000
2022-02-03 12:56:42.684	- INFO:  Triples converted: 23,500,000,000
2022-02-03 12:58:30.014	- INFO:  Triples converted: 23,600,000,000
2022-02-03 13:00:44.041	- INFO:  Triples converted: 23,700,000,000
2022-02-03 13:02:41.601	- INFO:  Triples converted: 23,800,000,000
2022-02-03 13:04:31.145	- INFO:  Triples converted: 23,900,000,000
2022-02-03 13:06:17.909	- INFO:  Triples converted: 24,000,000,000
2022-02-03 13:07:58.562	- INFO:  Triples converted: 24,100,000,000
2022-02-03 13:09:45.479	- INFO:  Triples converted: 24,200,000,000
2022-02-03 13:11:44.530	- INFO:  Triples converted: 24,300,000,000
2022-02-03 13:13:41.162	- INFO:  Triples converted: 24,400,000,000
2022-02-03 13:15:37.924	- INFO:  Triples converted: 24,500,000,000
2022-02-03 13:17:24.356	- INFO:  Triples converted: 24,600,000,000
2022-02-03 13:19:08.196	- INFO:  Triples converted: 24,700,000,000
2022-02-03 13:21:08.233	- INFO:  Triples converted: 24,800,000,000
2022-02-03 13:23:08.604	- INFO:  Triples converted: 24,900,000,000
2022-02-03 13:25:27.663	- INFO:  Triples converted: 25,000,000,000
2022-02-03 13:27:17.755	- INFO:  Triples converted: 25,100,000,000
2022-02-03 13:29:03.092	- INFO:  Triples converted: 25,200,000,000
2022-02-03 13:30:52.609	- INFO:  Triples converted: 25,300,000,000
2022-02-03 13:33:00.160	- INFO:  Triples converted: 25,400,000,000
2022-02-03 13:34:57.731	- INFO:  Triples converted: 25,500,000,000
2022-02-03 13:36:47.772	- INFO:  Triples converted: 25,600,000,000
2022-02-03 13:38:25.167	- INFO:  Triples converted: 25,700,000,000
2022-02-03 13:40:28.092	- INFO:  Triples converted: 25,800,000,000
2022-02-03 13:42:32.451	- INFO:  Triples converted: 25,900,000,000
2022-02-03 13:44:18.502	- INFO:  Triples converted: 26,000,000,000
2022-02-03 13:46:08.486	- INFO:  Triples converted: 26,100,000,000
2022-02-03 13:47:45.104	- INFO:  Triples converted: 26,200,000,000
2022-02-03 13:49:28.084	- INFO:  Triples converted: 26,300,000,000
2022-02-03 13:51:22.885	- INFO:  Triples converted: 26,400,000,000
2022-02-03 13:53:19.507	- INFO:  Triples converted: 26,500,000,000
2022-02-03 13:55:14.153	- INFO:  Triples converted: 26,600,000,000
2022-02-03 13:57:00.683	- INFO:  Triples converted: 26,700,000,000
2022-02-03 13:58:45.965	- INFO:  Triples converted: 26,800,000,000
2022-02-03 14:00:22.233	- INFO:  Triples converted: 26,900,000,000
2022-02-03 14:02:02.021	- INFO:  Triples converted: 27,000,000,000
2022-02-03 14:03:30.586	- INFO:  Done, total number of triples converted: 27,072,880,308
2022-02-03 14:03:30.791	- INFO:  Building prefix tree from internal vocabulary ...
2022-02-03 14:04:11.957	- INFO:  Words processed: 100,000,000
2022-02-03 14:05:17.470	- INFO:  Words processed: 200,000,000
2022-02-03 14:06:28.451	- INFO:  Words processed: 300,000,000
2022-02-03 14:07:43.608	- INFO:  Words processed: 400,000,000
2022-02-03 14:09:12.898	- INFO:  Words processed: 500,000,000
2022-02-03 14:10:26.170	- INFO:  Words processed: 600,000,000
2022-02-03 14:11:27.474	- INFO:  Words processed: 700,000,000
2022-02-03 14:13:15.041	- INFO:  Words processed: 800,000,000
2022-02-03 14:14:46.773	- INFO:  Words processed: 900,000,000
2022-02-03 14:15:17.044	- INFO:  Computing maximally compressing prefixes (greedy algorithm) ...
2022-02-03 15:16:53.604	- INFO:  Reduction of size of internal vocabulary: 46%
2022-02-03 15:18:21.559	- INFO:  Writing compressed vocabulary to disk ...
2022-02-03 15:40:47.936	- INFO:  Sorting for PSO permutation ...
2022-02-04 01:01:48.418	- INFO:  Removing duplicate triples ...
2022-02-04 11:00:37.645	- INFO:  Done, number of distinct triples is: 22,650,509,456
2022-02-04 11:00:37.646	- INFO:  Creating a pair of index permutations ... 
2022-02-04 19:48:26.759	- INFO:  Statistics for PSO: #relations = 59,862, #blocks = 44,995, #triples = 22,650,509,456
2022-02-04 19:48:26.777	- INFO:  Statistics for POS: #relations = 59,862, #blocks = 44,995, #triples = 22,650,509,456
2022-02-04 19:48:26.777	- INFO:  Exchanging multiplicities for PSO and POS ...
2022-02-04 19:48:26.803	- INFO:  Writing meta data for PSO and POS ...
2022-02-04 19:48:26.866	- INFO:  Sorting for SPO permutation ...
2022-02-05 04:18:41.117	- INFO:  Creating a pair of index permutations ... 
2022-02-05 10:39:07.231	- INFO:  Statistics for SPO: #relations = 2,683,434,680, #blocks = 28,807, #triples = 22,650,509,456
2022-02-05 10:39:07.231	- INFO:  Statistics for SOP: #relations = 2,683,434,680, #blocks = 28,807, #triples = 22,650,509,456
2022-02-05 10:39:07.231	- INFO:  Computing predicate patterns ...
2022-02-05 12:44:22.856	- INFO:  Number of patterns: 6,791,592
2022-02-05 12:44:22.856	- INFO:  Number of subjects with pattern: 2,683,434,680 [all]
2022-02-05 12:44:22.856	- INFO:  Total number of distinct subject-predicate pairs: 10,793,505,342
2022-02-05 12:44:22.856	- INFO:  Average number of predicates per subject: 4.0
2022-02-05 12:44:22.874	- INFO:  Average number of subjects per predicate: 243,146
2022-02-05 13:03:47.890	- INFO:  Exchanging multiplicities for SPO and SOP ...
2022-02-05 14:40:43.459	- INFO:  Writing meta data for SPO and SOP ...
2022-02-05 14:40:45.868	- INFO:  Sorting for OSP permutation ...
2022-02-05 23:26:18.899	- INFO:  Creating a pair of index permutations ... 
2022-02-06 06:46:43.742	- INFO:  Statistics for OSP: #relations = 3,046,826,131, #blocks = 38,121, #triples = 22,650,509,456
2022-02-06 06:46:43.746	- INFO:  Statistics for OPS: #relations = 3,046,826,131, #blocks = 38,121, #triples = 22,650,509,456
2022-02-06 06:46:43.746	- INFO:  Exchanging multiplicities for OSP and OPS ...
2022-02-06 08:02:47.855	- INFO:  Writing meta data for OSP and OPS ...
2022-02-06 08:02:50.547	- INFO:  Finished writing permutations
2022-02-06 08:02:50.570	- INFO:  Index build completed

Starting Server (3 min)

Starting the server failed on the 64 GB machine. I had to get myself a 128 GB machine - a used Dell PowerEdge R620 was the starting point. I had to copy the indexed data from the other machine which took some 2 1/2 h over a 1 GBit network. Then the server would start. During start the memory footprint went up to approx 75 GB and then down to 50 GB.

wf@sun:/hd/seel/qlever$ ./qlever -p -s
pulling qlever docker image started at Fr 11. Feb 20:46:42 CET 2022
Using default tag: latest
latest: Pulling from adfreiburg/qlever
Digest: sha256:984cf99ac0b2b804fd7035ff15e2362acd19af6c58b653a4aac3e52e7273b2cd
Status: Image is up to date for adfreiburg/qlever:latest
docker.io/adfreiburg/qlever:latest
pulling qlever docker image finished at Fr 11. Feb 20:46:43 CET 2022 after 1 seconds
2022-02-11 19:46:44.074	- INFO:  QLever Server, compiled on Feb  3 2022 19:24:25
2022-02-11 19:46:44.075	- INFO:  Initializing server ...
2022-02-11 19:46:44.096	- INFO:  Reading internal vocabulary from file /index/wikidata.vocabulary ...
2022-02-11 19:47:15.816	- INFO:  Done, number of words: 938,491,990
2022-02-11 19:47:15.819	- INFO:  Number of words in external vocabulary: 2,289,883,113
2022-02-11 19:47:15.849	- INFO:  Registered PSO permutation: #relations = 59,862, #blocks = 44,995, #triples = 22,650,509,456
2022-02-11 19:47:15.872	- INFO:  Registered POS permutation: #relations = 59,862, #blocks = 44,995, #triples = 22,650,509,456
2022-02-11 19:47:15.881	- INFO:  Registered OPS permutation: #relations = 3,046,826,131, #blocks = 38,121, #triples = 22,650,509,456
2022-02-11 19:47:15.888	- INFO:  Registered OSP permutation: #relations = 3,046,826,131, #blocks = 38,121, #triples = 22,650,509,456
2022-02-11 19:47:15.896	- INFO:  Registered SPO permutation: #relations = 2,683,434,680, #blocks = 28,807, #triples = 22,650,509,456
2022-02-11 19:47:15.903	- INFO:  Registered SOP permutation: #relations = 2,683,434,680, #blocks = 28,807, #triples = 22,650,509,456
2022-02-11 19:47:15.903	- INFO:  Reading patterns from file /index/wikidata.index.patterns ...
2022-02-11 19:49:20.619	- INFO:  Sorting random result tables to estimate the sorting performance of this machine ...
2022-02-11 19:50:41.147	- INFO:  The server is ready

Issues

see