EMWConTalk2023-04

EMWConTalk2023-04/<ul><li>"Expression error: Unrecognized word "wolfgangfahl"." is not a number.</li> <li>"WolfgangFahl" is not a number.</li></ul>

BITPlan GmbH
	Inception: 1999   Focus on Software-Engineering   Training, Consulting, Projects …   Switch from Lotus Notes to  Semantic Mediawiki in 2013   ProfiWiki since 2015
Copyright (c) 2015-2023 BITPlan GmbH

Agenda
	Wikidata Synchronization
Copyright (c) 2015-2023 BITPlan GmbH

SMW and Wikidata storage

Both systems use triples: subject, predicate and object
It's a girl revisited:

It's a girl!

The princess of cambridge was born on [[is born::2015-05-03]] 
Her rank inline to the throne is [[throne rank is::4]].
Her mother is [[mother is::Duchess Kate]].

SMW and Wikidata as databases?
	Compare to a relational database: Tables are dynamically created via SMW ask-queries e.g.List of Persons Wikidata SPARQL queries e.g Recent deaths in the royal family A page corresponds to a row in a table e.g. SMW Page Elizabeth II e.g. Wikidata entry Elizabeth II A property corresponds to a column in a table e.g. SMW Property Property:Person died e.g. Wikidata Property date of death
Copyright (c) 2015-2023 BITPlan GmbH

SMW relational operations

Copyright (c) 2015-2023 BITPlan GmbH

Row and Column definition operations

Caption text
Operation	Manually	Via Api	Via Wikipush
Create	create page and use markup to set property ✓	edit ✓	wikipush, wikiupload, wikirestore ✓
Read	visit link of page ✓	read html or markup {Done}}	wikibackup, wikipush, wikiedit ✓
Update	modify page via edit	edit ✓	wikiedit, wikirestore, wikipush ✓
Delete	delete page ✓	delete {Done}}	wikinuke ✓

Column value operations

Caption text
Operation	Manually	Via Api	Via Wikipush
Create	add property value manually on a page ✓	❌	❌ -> ✓
Read	use ask queries to retrieve property values ✓	edit ✓	wikiquery ✓
Update	modify property value manually on a page ✓	❌	❌ -> ✓
Delete	remove property value manually on a page ✓	❌	❌

Modifying a Single triple

wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark 
editing 1 pages in family (dry run)
1/1 (100.00%): editing Alexandra_of_Denmark ...👍 |childOf=F74
 |parentOf=F2
 |spouse=Q20875
+|label=Alexandra of Denmark
 }}
 [[Category:frontend]]
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark -f
editing 1 pages in family (forced)
1/1 (100.00%): editing Alexandra_of_Denmark ...✅

Example Usecases
	Royal Family Scholary communication
Copyright (c) 2015-2023 BITPlan GmbH

Example Royal Family

BITPlan's Royal Family wiki

Sync Royal Family

http://family.bitplan.com/index.php/List_of_Persons

smwsync -u -t family --context FamilyContext --topic Person -p desc label died died_at --progress
updating cache for FamilyContext:Person from wiki family ...
stored 59 Person items to /Users/wf/.smwsync/family/FamilyContext/Person.json
59 Person items to sync ...
William IV of the United Kingdom→died_at: 100%|█████| 236/236 [01:48<00:00,  2.18it/s]

Example: Scholary communication
	https://cr.bitplan.com
Copyright (c) 2015-2023 BITPlan GmbH

Sync Event Series

https://cr.bitplan.com/index.php/List_of_EventSeries

smwsync -u -t cr --topic EventSeries -p desc title home --progress
updating cache for CrSchema:EventSeries from wiki cr ...
stored 10 EventSeries items to /Users/wf/.smwsync/cr/CrSchema/EventSeries.json
10 EventSeries items to sync ...
SGAI→homepage: 100%|██████████████████████████████████| 30/30 [00:28<00:00,  1.05it/s]

Sync Institutions

https://cr.bitplan.com/index.php/List_of_Institutions

smwsync -u -t cr --topic Institution -p desc home --progress
updating cache for CrSchema:Institution from wiki cr ...
stored 4 Institution items to /Users/wf/.smwsync/cr/CrSchema/Institution.json
4 Institution items to sync ...
Universität Stuttgart→homepage: 100%|█████████████████| 12/12 [00:12<00:00,  1.06s/it]

How SMWSync works
	Self describing Properties MetaModel, SiDIF, LinkML Mapping SMW Properties to Wikidata properties Command Line Tool
Copyright (c) 2015-2023 BITPlan GmbH

Self describing Properties
	2013: Pierre Racine 2015: BITPlan ProfiWiki Person Label (to be synced with wikidata)
Copyright (c) 2015-2023 BITPlan GmbH

MetaModel, SiDIF, LinkML

ypgen --wikiId family --context FamilyContext --serve --host localhost

FamilyContext SiDIF is in legacy SiDIF triple format.

pyMetaModel mm_cmd command line tool will generate LinkML yaml markup and other target formats. LinkML can then generate even form format see:

pyMetaModel family example files

Mapping SMW Properties to Wikidata properties ==

# list of property mappings
# for family  versus wikidata
- topic: Person
  prop_list:
  - pid: qid
    smw_prop: qid
  - arg: label
    pid: label
    smw_prop: label
  - arg: desc
    pid: description
    pid_label: description
    smw_prop: description
  - arg: died
    pid: P570
    pid_label: date of death
    smw_prop: died
  - arg: died_at
    pid: P20
    pid_label: place of death
    smw_prop: died_at

Command Line Tool

https://wiki.bitplan.com/index.php/PySMWSync#Command_line

usage: smwsync [-h] [-a] [--context CONTEXT] [-cpm CREATEPROPERTYMAP] [-d] [--dry]
               [-e ENDPOINT] [--progress] [-p PROPS [PROPS ...]] [--proplist]
               [-pm PROPERTYMAP] [-pk PRIMARYKEY]
               [-pkv PRIMARYKEYVALUES [PRIMARYKEYVALUES ...]] [-t TARGET] [-u]
               [--topic TOPIC] [--verbose] [-V]

Todos
	synchronization of complete entities or sets of properties (--all option for properties) detection of sync conflicts bidirectional sync publish to wikidata from smw GUI Sync dialog generation of smw topics and properties from wikidata mapping information in property definition page support of intext annotation see https://www.semantic-mediawiki.org/wiki/Help:In-text_annotation
Copyright (c) 2015-2023 BITPlan GmbH

Links
	https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki https://www.wikidata.org/wiki/Wikidata:Main_Page https://wiki.bitplan.com/index.php/Py-3rdparty-mediawiki https://wiki.bitplan.com/index.php/PySMWSync https://wiki.bitplan.com/index.php/PyMetaModel https://wiki.bitplan.com/index.php/Py-yprinciple-gen https://linkml.io/
Copyright (c) 2015-2023 BITPlan GmbH

Questions / Discussion

Copyright (c) 2015-2023 BITPlan GmbH

Example: Scholary communication

https://cr.bitplan.com

Sync Event Series

https://cr.bitplan.com/index.php/List_of_EventSeries

smwsync -u -t cr --topic EventSeries -p desc title home --progress
updating cache for CrSchema:EventSeries from wiki cr ...
stored 10 EventSeries items to /Users/wf/.smwsync/cr/CrSchema/EventSeries.json
10 EventSeries items to sync ...
SGAI→homepage: 100%|██████████████████████████████████| 30/30 [00:28<00:00,  1.05it/s]

Sync Institutions

https://cr.bitplan.com/index.php/List_of_Institutions

smwsync -u -t cr --topic Institution -p desc home --progress
updating cache for CrSchema:Institution from wiki cr ...
stored 4 Institution items to /Users/wf/.smwsync/cr/CrSchema/Institution.json
4 Institution items to sync ...
Universität Stuttgart→homepage: 100%|█████████████████| 12/12 [00:12<00:00,  1.06s/it]

How SMWSync works

Self describing Properties
MetaModel, SiDIF, LinkML
Mapping SMW Properties to Wikidata properties
Command Line Tool

Self describing Properties

MetaModel, SiDIF, LinkML

ypgen --wikiId family --context FamilyContext --serve --host localhost

FamilyContext SiDIF is in legacy SiDIF triple format.

pyMetaModel mm_cmd command line tool will generate LinkML yaml markup and other target formats. LinkML can then generate even form format see:

pyMetaModel family example files

Mapping SMW Properties to Wikidata properties

# list of property mappings
# for family  versus wikidata
- topic: Person
  prop_list:
  - pid: qid
    smw_prop: qid
  - arg: label
    pid: label
    smw_prop: label
  - arg: desc
    pid: description
    pid_label: description
    smw_prop: description
  - arg: died
    pid: P570
    pid_label: date of death
    smw_prop: died
  - arg: died_at
    pid: P20
    pid_label: place of death
    smw_prop: died_at

Command Line Tool

https://wiki.bitplan.com/index.php/PySMWSync#Command_line

usage: smwsync [-h] [-a] [--context CONTEXT] [-cpm CREATEPROPERTYMAP] [-d] [--dry]
               [-e ENDPOINT] [--progress] [-p PROPS [PROPS ...]] [--proplist]
               [-pm PROPERTYMAP] [-pk PRIMARYKEY]
               [-pkv PRIMARYKEYVALUES [PRIMARYKEYVALUES ...]] [-t TARGET] [-u]
               [--topic TOPIC] [--verbose] [-V]

Todos

synchronization of complete entities or sets of properties (--all option for properties)
- detection of sync conflicts
bidirectional sync
- publish to wikidata from smw
GUI
- Sync dialog
generation of smw topics and properties from wikidata
mapping information in property definition page
support of intext annotation
- see https://www.semantic-mediawiki.org/wiki/Help:In-text_annotation

Links

Questions / Discussion

EMWConTalk2023-04

Example: Scholary communication

Sync Event Series

Sync Institutions

How SMWSync works

Self describing Properties

MetaModel, SiDIF, LinkML

Mapping SMW Properties to Wikidata properties

Command Line Tool

Todos

Links

Questions / Discussion

Navigation menu

Search