EMWConTalk2023-04

view EMWConTalk2023-04 as slides

Wolfgang Fahl

Wolfgang Fahl
	Computer Scientist Expert Trainer for Software Architecture and Requirements Engineering PhD Candidate at RWTH Aachen i5
Copyright (c) 2015-2023 BITPlan GmbH

BITPlan GmbH

BITPlan GmbH
	Inception: 1999   Focus on Software-Engineering   Training, Consulting, Projects …   Switch from Lotus Notes to  Semantic Mediawiki in 2013   ProfiWiki since 2015
Copyright (c) 2015-2023 BITPlan GmbH

Agenda

Agenda
	Wikidata Synchronization
Copyright (c) 2015-2023 BITPlan GmbH

SMW and Wikidata storage

Both systems use triples: subject, predicate and object
It's a girl revisited:

It's a girl!

The princess of cambridge was born on [[is born::2015-05-03]] 
Her rank inline to the throne is [[throne rank is::4]].
Her mother is [[mother is::Duchess Kate]].

SMW and Wikidata as databases?

Compare to a relational database:

Tables are dynamically created via
- SMW ask-queries e.g.List of Persons
- Wikidata SPARQL queries e.g Recent deaths in the royal family
A page corresponds to a row in a table
- e.g. SMW Page Elizabeth II
- e.g. Wikidata entry Elizabeth II
A property corresponds to a column in a table
- e.g. SMW Property Property:Person died
- e.g. Wikidata Property date of death

SMW relational operations

Row and Column definition operations

Caption text
Operation	Manually	Via Api	Via Wikipush
Create	create page and use markup to set property ✓	edit ✓	wikipush, wikiupload, wikirestore ✓
Read	visit link of page ✓	read html or markup {Done}}	wikibackup, wikipush, wikiedit ✓
Update	modify page via edit	edit ✓	wikiedit, wikirestore, wikipush ✓
Delete	delete page ✓	delete {Done}}	wikinuke ✓

Column value operations

Caption text
Operation	Manually	Via Api	Via Wikipush
Create	add property value manually on a page ✓	❌	❌ -> ✓
Read	use ask queries to retrieve property values ✓	edit ✓	wikiquery ✓
Update	modify property value manually on a page ✓	❌	❌ -> ✓
Delete	remove property value manually on a page ✓	❌	❌

Modifying a Single triple

wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark 
editing 1 pages in family (dry run)
1/1 (100.00%): editing Alexandra_of_Denmark ...👍 |childOf=F74
 |parentOf=F2
 |spouse=Q20875
+|label=Alexandra of Denmark
 }}
 [[Category:frontend]]
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark -f
editing 1 pages in family (forced)
1/1 (100.00%): editing Alexandra_of_Denmark ...✅

Example Usecases

Royal Family
Scholary communication

Example Royal Family

BITPlan's Royal Family wiki]

Sync Royal Family

http://family.bitplan.com/index.php/List_of_Persons

smwsync -u -t family --context FamilyContext --topic Person -p desc label died died_at --progress
updating cache for FamilyContext:Person from wiki family ...
stored 59 Person items to /Users/wf/.smwsync/family/FamilyContext/Person.json
59 Person items to sync ...
William IV of the United Kingdom→died_at: 100%|█████| 236/236 [01:48<00:00,  2.18it/s]

Example: Scholary communication

https://cr.bitplan.com

Sync Event Series

https://cr.bitplan.com/index.php/List_of_EventSeries

smwsync -u -t cr --topic EventSeries -p desc title home --progress
updating cache for CrSchema:EventSeries from wiki cr ...
stored 10 EventSeries items to /Users/wf/.smwsync/cr/CrSchema/EventSeries.json
10 EventSeries items to sync ...
SGAI→homepage: 100%|██████████████████████████████████| 30/30 [00:28<00:00,  1.05it/s]

Sync Institutions

https://cr.bitplan.com/index.php/List_of_Institutions

smwsync -u -t cr --topic Institution -p desc home --progress
updating cache for CrSchema:Institution from wiki cr ...
stored 4 Institution items to /Users/wf/.smwsync/cr/CrSchema/Institution.json
4 Institution items to sync ...
Universität Stuttgart→homepage: 100%|█████████████████| 12/12 [00:12<00:00,  1.06s/it]

How SMWSync works

Self describing Properties
MetaModel, SiDIF, LinkML
Mapping SMW Properties to Wikidata properties
Command Line Tool

Self describing Properties

MetaModel, SiDIF, LinkML

ypgen --wikiId family --context FamilyContext --serve --host localhost

FamilyContext SiDIF is in legacy SiDIF triple format.

pyMetaModel mm_cmd command line tool will generate LinkML yaml markup and other target formats. LinkML can then generate even form format see:

pyMetaModel family example files

Mapping SMW Properties to Wikidata properties

# list of property mappings
# for family  versus wikidata
- topic: Person
  prop_list:
  - pid: qid
    smw_prop: qid
  - arg: label
    pid: label
    smw_prop: label
  - arg: desc
    pid: description
    pid_label: description
    smw_prop: description
  - arg: died
    pid: P570
    pid_label: date of death
    smw_prop: died
  - arg: died_at
    pid: P20
    pid_label: place of death
    smw_prop: died_at

Command Line Tool

https://wiki.bitplan.com/index.php/PySMWSync#Command_line

usage: smwsync [-h] [-a] [--context CONTEXT] [-cpm CREATEPROPERTYMAP] [-d] [--dry]
               [-e ENDPOINT] [--progress] [-p PROPS [PROPS ...]] [--proplist]
               [-pm PROPERTYMAP] [-pk PRIMARYKEY]
               [-pkv PRIMARYKEYVALUES [PRIMARYKEYVALUES ...]] [-t TARGET] [-u]
               [--topic TOPIC] [--verbose] [-V]

Todos

synchronization of complete entities or sets of properties (--all option for properties)
- detection of sync conflicts
bidirectional sync
- publish to wikidata from smw
GUI
- Sync dialog
generation of smw topics and properties from wikidata
mapping information in property definition page
support of intext annotation
- see https://www.semantic-mediawiki.org/wiki/Help:In-text_annotation

Links

Questions / Discussion