EMWConTalk2023-04

Wolfgang Fahl
	Computer Scientist Expert Trainer for Software Architecture and Requirements Engineering PhD Candidate at RWTH Aachen i5
Copyright (c) 2015-2023 BITPlan GmbH

BITPlan GmbH
	Inception: 1999   Focus on Software-Engineering   Training, Consulting, Projects …   Switch from Lotus Notes to  Semantic Mediawiki in 2013   ProfiWiki since 2015
Copyright (c) 2015-2023 BITPlan GmbH

Agenda
	Wikidata Synchronization
Copyright (c) 2015-2023 BITPlan GmbH

{

SMW and Wikidata storage

Both systems use triples: subject, predicate and object
It's a girl revisited:

It's a girl!

The princess of cambridge was born on [[is born::2015-05-03]] 
Her rank inline to the throne is [[throne rank is::4]].
Her mother is [[mother is::Duchess Kate]].

{

SMW and Wikidata as databases?
	Compare to a relational database: Tables are dynamically created via SMW ask-queries e.g.List of Persons Wikidata SPARQL queries e.g Recent deaths in the royal family A page corresponds to a row in a table e.g. SMW Page Elizabeth II e.g. Wikidata entry Elizabeth II A property corresponds to a column in a table e.g. SMW Property Property:Person died e.g. Wikidata Property date of death
Copyright (c) 2015-2023 BITPlan GmbH

{

SMW relational operations

Copyright (c) 2015-2023 BITPlan GmbH

Row and Column definition operations

Caption text
Operation	Manually	Via Api	Via Wikipush
Create	create page and use markup to set property ✓	edit ✓	wikipush, wikiupload, wikirestore ✓
Read	visit link of page ✓	read html or markup {Done}}	wikibackup, wikipush, wikiedit ✓
Update	modify page via edit	edit ✓	wikiedit, wikirestore, wikipush ✓
Delete	delete page ✓	delete {Done}}	wikinuke ✓

Column value operations

Caption text
Operation	Manually	Via Api	Via Wikipush
Create	add property value manually on a page ✓	❌	❌ -> ✓
Read	use ask queries to retrieve property values ✓	edit ✓	wikiquery ✓
Update	modify property value manually on a page ✓	❌	❌ -> ✓
Delete	remove property value manually on a page ✓	❌	❌

{

Modifying a Single triple

py-3rdparty-mediawiki issue #91 wikiedit allow to modify single parameter of a macro WikiSon see WikiSon
http://family.bitplan.com/index.php/Alexandra_of_Denmark

wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark 
editing 1 pages in family (dry run)
1/1 (100.00%): editing Alexandra_of_Denmark ...👍 |childOf=F74
 |parentOf=F2
 |spouse=Q20875
+|label=Alexandra of Denmark
 }}
 [[Category:frontend]]
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark -f
editing 1 pages in family (forced)
1/1 (100.00%): editing Alexandra_of_Denmark ...✅

{

Example Usecases
	Royal Family Scholary communication
Copyright (c) 2015-2023 BITPlan GmbH

{

Example Royal Family

BITPlan's Royal Family wiki

{

Sync Royal Family

http://family.bitplan.com/index.php/List_of_Persons

smwsync -u -t family --context FamilyContext --topic Person -p desc label died died_at --progress
updating cache for FamilyContext:Person from wiki family ...
stored 59 Person items to /Users/wf/.smwsync/family/FamilyContext/Person.json
59 Person items to sync ...
William IV of the United Kingdom→died_at: 100%|█████| 236/236 [01:48<00:00,  2.18it/s]

{

Example: Scholary communication
	https://cr.bitplan.com
Copyright (c) 2015-2023 BITPlan GmbH

{

Sync Event Series

https://cr.bitplan.com/index.php/List_of_EventSeries

smwsync -u -t cr --topic EventSeries -p desc title home --progress
updating cache for CrSchema:EventSeries from wiki cr ...
stored 10 EventSeries items to /Users/wf/.smwsync/cr/CrSchema/EventSeries.json
10 EventSeries items to sync ...
SGAI→homepage: 100%|██████████████████████████████████| 30/30 [00:28<00:00,  1.05it/s]

{

Sync Institutions

https://cr.bitplan.com/index.php/List_of_Institutions

smwsync -u -t cr --topic Institution -p desc home --progress
updating cache for CrSchema:Institution from wiki cr ...
stored 4 Institution items to /Users/wf/.smwsync/cr/CrSchema/Institution.json
4 Institution items to sync ...
Universität Stuttgart→homepage: 100%|█████████████████| 12/12 [00:12<00:00,  1.06s/it]

{

How SMWSync works
	Self describing Properties MetaModel, SiDIF, LinkML Mapping SMW Properties to Wikidata properties Command Line Tool
Copyright (c) 2015-2023 BITPlan GmbH

{

Self describing Properties
	2013: Pierre Racine 2015: BITPlan ProfiWiki Person Label (to be synced with wikidata)
Copyright (c) 2015-2023 BITPlan GmbH

{

MetaModel, SiDIF, LinkML

ypgen --wikiId family --context FamilyContext --serve --host localhost

FamilyContext SiDIF is in legacy SiDIF triple format.

pyMetaModel mm_cmd command line tool will generate LinkML yaml markup and other target formats. LinkML can then generate even form format see:

pyMetaModel family example files

{

Mapping SMW Properties to Wikidata properties ==

# list of property mappings
# for family  versus wikidata
- topic: Person
  prop_list:
  - pid: qid
    smw_prop: qid
  - arg: label
    pid: label
    smw_prop: label
  - arg: desc
    pid: description
    pid_label: description
    smw_prop: description
  - arg: died
    pid: P570
    pid_label: date of death
    smw_prop: died
  - arg: died_at
    pid: P20
    pid_label: place of death
    smw_prop: died_at

{

Command Line Tool

https://wiki.bitplan.com/index.php/PySMWSync#Command_line

usage: smwsync [-h] [-a] [--context CONTEXT] [-cpm CREATEPROPERTYMAP] [-d] [--dry]
               [-e ENDPOINT] [--progress] [-p PROPS [PROPS ...]] [--proplist]
               [-pm PROPERTYMAP] [-pk PRIMARYKEY]
               [-pkv PRIMARYKEYVALUES [PRIMARYKEYVALUES ...]] [-t TARGET] [-u]
               [--topic TOPIC] [--verbose] [-V]

{

Todos
	synchronization of complete entities or sets of properties (--all option for properties) detection of sync conflicts bidirectional sync publish to wikidata from smw GUI Sync dialog generation of smw topics and properties from wikidata mapping information in property definition page support of intext annotation see https://www.semantic-mediawiki.org/wiki/Help:In-text_annotation
Copyright (c) 2015-2023 BITPlan GmbH

{

Links
	https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki https://www.wikidata.org/wiki/Wikidata:Main_Page https://wiki.bitplan.com/index.php/Py-3rdparty-mediawiki https://wiki.bitplan.com/index.php/PySMWSync https://wiki.bitplan.com/index.php/PyMetaModel https://wiki.bitplan.com/index.php/Py-yprinciple-gen https://linkml.io/
Copyright (c) 2015-2023 BITPlan GmbH

{

Questions / Discussion

Copyright (c) 2015-2023 BITPlan GmbH