Difference between revisions of "EMWConTalk2023-04"

From BITPlan Wiki
Jump to navigation Jump to search
 
(40 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{PageSequence|prev=SMWConTalk2022-10|next=|category=talk|categoryIcon=picture-o}}{{ShowSlides}}
+
{{PageSequence|prev=SMWConTalk2022-10|next=SMWConTalk2024-11|category=talk|categoryIcon=picture-o}}
 
__NOTOC__
 
__NOTOC__
== Wolfgang Fahl ==
+
{{NextSlide|1|WolfgangFahl}}
{{font-size|[[File:WolfgangFahl.png|200px|left|link=https://www.semantic-mediawiki.org/wiki/User:WolfgangFahl|left]]
+
{{NextSlide|2|BITPlanGmbH}}
* Computer Scientist
+
{{NextSlide|3|Agenda}}
* [[File:20210702 Semantic MediaWiki logo with wordmark.png|128px]] Expert
+
{{{NextSlide|4|SMWWikidataStorage}}
* Trainer for Software Architecture and Requirements Engineering
+
{{{NextSlide|5|SMWWikidataDatabases}}
* PhD Candidate at [https://www.wikidata.org/wiki/Q117301965 RWTH Aachen i5]
+
{{{NextSlide|6|SMWRelationalOperators}}
}}
+
{{{NextSlide|7|ModifyingASingleTriple}}
 
+
{{{NextSlide|8|ExampleUseCases}}
== BITPlan GmbH ==
+
{{{NextSlide|9|ExampleRoyalFamily}}
[[File:BITPlanLogoFontLessTransparent.png|240px|link=http://www.bitplan.com|BITPlan]]
+
{{{NextSlide|10|SyncRoyalFamily}}
* Inception: 1999


+
{{{NextSlide|11|ExampleScholarlyCommunication}}
* Focus on Software-Engineering 

Training, Consulting, Projects …
+
{{{NextSlide|12|SyncEventSeries}}
* 

Switch from Lotus Notes to 
Semantic Mediawiki in 2013


+
{{{NextSlide|13|SyncInstitutions}}
* [http://profiwiki.bitplan.com ProfiWiki] since 2015
+
{{{NextSlide|14|HowSMWSyncWorks}}
 
+
{{{NextSlide|15|SelfDescribingProperties}}
== Agenda ==
+
{{{NextSlide|16|MetaModelSiDIFandLinkMl}}
[[File:Wikidata-logo-en.svg|128px]] Wikidata
+
{{{NextSlide|17|MappingSMWAndWikiDAtaProperties}}
 
+
{{{NextSlide|18|CommandLineTool}}
[[File:20210702 Semantic MediaWiki logo with wordmark.png|128px]]
+
{{{NextSlide|19|Todos}}
 
+
{{{NextSlide|20|Links}}
[[File:Octicons-sync.svg|128px]] Synchronization
+
{{{NextSlide|21|QuestionsDiscussion}}
 
 
== SMW and Wikidata storage ==
 
* Both systems use triples: subject, predicate and object
 
* It's a girl revisited: [[File:Royalbaby.png|right|frame|It's a girl!]]
 
<pre style='font-size:24px'>
 
The princess of cambridge was born on [[is born::2015-05-03]]
 
Her rank inline to the throne is [[throne rank is::4]].
 
Her mother is [[mother is::Duchess Kate]].
 
</pre>
 
* http://royal-family.bitplan.com/index.php/Princess_Charlotte_of_Cambridge
 
* http://family.bitplan.com/index.php/Princess_Charlotte_of_Cambridge
 
 
 
== SMW and Wikidata as databases? ==
 
Compare to a relational database:
 
 
 
* Tables are dynamically created via
 
** SMW ask-queries e.g.[http://family.bitplan.com/index.php/List_of_Persons List of Persons]
 
** Wikidata SPARQL queries e.g [https://w.wiki/6bxU Recent deaths in the royal family]
 
* A page corresponds to a row in a table
 
** e.g. SMW Page [http://family.bitplan.com/index.php/Elizabeth_II Elizabeth II]
 
** e.g. Wikidata entry [https://www.wikidata.org/wiki/Q9682]
 
* A property corresponds to a column in a table
 
** e.g. SMW Property [http://family.bitplan.com/index.php/Property:Person_died Property:Person died]
 
** e.g. Wikidata Property [https://www.wikidata.org/wiki/Property:P570 date of death]
 
 
 
== SMW relational operations ==
 
'''Row and Column definition operations'''
 
 
 
{| class="wikitable"
 
|+ Caption text
 
|-
 
! Operation !! Manually !! Via Api !! Via Wikipush
 
|-
 
| Create || create page and use markup to set property {{Done}} || edit {{Done}} || wikipush, wikiupload, wikirestore {{Done}}
 
|-
 
| Read || visit link of page {{Done}} || read html or markup {Done}} || wikibackup, wikipush, wikiedit {{Done}}
 
|-
 
| Update || modify page via edit|| [https://www.mediawiki.org/wiki/API:Edit edit] {{Done}} || wikiedit, wikirestore, wikipush {{Done}}
 
|-
 
| Delete || delete page {{Done}} || delete {Done}} || wikinuke {{Done}}
 
|}
 
'''Column value operations'''
 
{| class="wikitable"
 
|+ Caption text
 
|-
 
! Operation !! Manually !! Via Api !! Via Wikipush
 
|-
 
| Create || add property value manually on a page {{Done}} || {{UtfIcon|utf=❌|size=24|color=red}} || {{UtfIcon|utf=❌|size=24|color=red}} -> {{Done}}
 
|-
 
| Read || use ask queries to retrieve property values {{Done}} || edit {{Done}} || wikiquery {{Done}}
 
|-
 
| Update || modify property value manually on a page {{Done}} || {{UtfIcon|utf=❌|size=24|color=red}} || {{UtfIcon|utf=❌|size=24|color=red}} -> {{Done}}
 
|-
 
| Delete || remove property value manually on a page {{Done}} ||  {{UtfIcon|utf=❌|size=24|color=red}} || {{UtfIcon|utf=❌|size=24|color=red}}
 
|}
 
 
 
== Modifying a Single triple ==
 
* [https://github.com/WolfgangFahl/py-3rdparty-mediawiki/issues/91 py-3rdparty-mediawiki issue #91 wikiedit allow to modify single parameter of a macro WikiSon ] see [[WikiSon]]
 
* http://family.bitplan.com/index.php/Alexandra_of_Denmark
 
 
 
<source lang='bash' highlight='1,9' style='font-size:24px'>
 
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark
 
editing 1 pages in family (dry run)
 
1/1 (100.00%): editing Alexandra_of_Denmark ...👍 |childOf=F74
 
|parentOf=F2
 
|spouse=Q20875
 
+|label=Alexandra of Denmark
 
}}
 
[[Category:frontend]]
 
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark -f
 
editing 1 pages in family (forced)
 
1/1 (100.00%): editing Alexandra_of_Denmark ...✅
 
</source>
 
 
 
== Example Usecases ==
 
* Royal Family
 
* Scholary communication
 
== Example Royal Family ==
 
https://upload.wikimedia.org/wikipedia/commons/thumb/e/eb/Royal_Coat_of_Arms_of_the_United_Kingdom_%28Tudor_crown%29.svg/257px-Royal_Coat_of_Arms_of_the_United_Kingdom_%28Tudor_crown%29.svg.png
 
 
 
* [http://royal-family.bitplan.com/index.php/Main_Page BITPlan's Royal Family wiki]]
 
 
 
* http://diagrams.bitplan.com/render/png/0xe4ab021a.png
 
 
 
== Sync Royal Family ==
 
* http://family.bitplan.com/index.php/List_of_Persons
 
 
 
<source lang='bash' highlight='1'>
 
smwsync -u -t family --context FamilyContext --topic Person -p desc label died died_at --progress
 
updating cache for FamilyContext:Person from wiki family ...
 
stored 59 Person items to /Users/wf/.smwsync/family/FamilyContext/Person.json
 
59 Person items to sync ...
 
William IV of the United Kingdom→died_at: 100%|█████| 236/236 [01:48<00:00,  2.18it/s]
 
</source>
 
* http://royal-family.bitplan.com/index.php/Elizabeth_II
 
* http://family.bitplan.com/index.php/Elizabeth_II
 
 
 
== Example: Scholary communication ==
 
 
 
http://diagrams.bitplan.com/render/png/0x1a095a5a.png
 
* https://cr.bitplan.com
 
 
 
 
 
== Sync Institutions ==
 
* https://ceur-ws.bitplan.com/index.php/List_of_Institutions
 
<source lang='bash' highlight='1'>
 
smwsync -u -t cr --topic EventSeries -p desc title home --progress
 
updating cache for CrSchema:EventSeries from wiki cr ...
 
stored 10 EventSeries items to /Users/wf/.smwsync/cr/CrSchema/EventSeries.json
 
10 EventSeries items to sync ...
 
SGAI→homepage: 100%|██████████████████████████████████| 30/30 [00:28<00:00,  1.05it/s]
 
</source>
 
 
 
== How SMWSync works ==
 
* Self describing Properties
 
* MetaModel, SiDIF, LinkML
 
* Mapping SMW Properties to Wikidata properties
 
* Command Line Tool
 
 
 
== Self describing Properties ==
 
 
 
* [https://www.semantic-mediawiki.org/wiki/Metadata_management#Pierre_Racine 2013: Pierr Racine]
 
* [https://www.semantic-mediawiki.org/wiki/Metadata_management#ProfiWiki 2015: BITPlan ProfiWiki]
 
* [http://family.bitplan.com/index.php/Property:Person_label Person Label (to be synced with wikidata)]
 
 
 
== MetaModel, SiDIF, LinkML ==
 
[[File:YpgenScreenShotFamilyContext2023-04-19.png|600px]]
 
<source lang='bash'>
 
ypgen --wikiId family --context FamilyContext --serve --host localhost
 
</source>
 
 
 
[http://family.bitplan.com/index.php/FamilyContext#tab=sidif FamilyContext SiDIF] is in legacy [[SiDIF]] triple format.
 
 
 
[https://github.com/WolfgangFahl/pyMetaModel pyMetaModel mm_cmd command line tool] will generate LinkML yaml markup and other target formats.
 
LinkML can then generate even form format see:
 
 
 
[https://github.com/WolfgangFahl/pyMetaModel/tree/main/examples/family pyMetaModel family example files]
 
 
 
== Mapping SMW Properties to Wikidata properties ==
 
<source lang='yaml'>
 
# list of property mappings
 
# for family  versus wikidata
 
- topic: Person
 
  prop_list:
 
  - pid: qid
 
    smw_prop: qid
 
  - arg: label
 
    pid: label
 
    smw_prop: label
 
  - arg: desc
 
    pid: description
 
    pid_label: description
 
    smw_prop: description
 
  - arg: died
 
    pid: P570
 
    pid_label: date of death
 
    smw_prop: died
 
  - arg: died_at
 
    pid: P20
 
    pid_label: place of death
 
    smw_prop: died_at
 
</source>
 
 
 
== Command Line Tool ==
 
* https://wiki.bitplan.com/index.php/PySMWSync#Command_line
 
<source lang='bash'>
 
usage: smwsync [-h] [-a] [--context CONTEXT] [-cpm CREATEPROPERTYMAP] [-d] [--dry]
 
              [-e ENDPOINT] [--progress] [-p PROPS [PROPS ...]] [--proplist]
 
              [-pm PROPERTYMAP] [-pk PRIMARYKEY]
 
              [-pkv PRIMARYKEYVALUES [PRIMARYKEYVALUES ...]] [-t TARGET] [-u]
 
              [--topic TOPIC] [--verbose] [-V]
 
</source>
 
== Todos ==
 
* synchronization of complete entities or sets of properties (--all option for properties)
 
** detection of sync conflicts
 
* bidirectional sync
 
** publish to wikidata from smw
 
* GUI
 
** Sync dialog
 
* generation of smw topics and properties from wikidata
 
* mapping information in property definition page
 
* support of intext annotation
 
** see https://www.semantic-mediawiki.org/wiki/Help:In-text_annotation
 
 
 
== Links ==
 
* https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki
 
* https://www.wikidata.org/wiki/Wikidata:Main_Page
 
* https://wiki.bitplan.com/index.php/Py-3rdparty-mediawiki
 
* https://wiki.bitplan.com/index.php/PySMWSync
 
* https://wiki.bitplan.com/index.php/PyMetaModel
 
* https://wiki.bitplan.com/index.php/Py-yprinciple-gen
 
* https://linkml.io/
 
 
 
== Questions / Discussion ==
 
[[File:Bitplanpartner.png]]
 

Latest revision as of 11:09, 4 November 2024


BITPlanLogo2012 138x77.png
Wolfgang Fahl
WolfgangFahl.png
  • Computer Scientist
  • 20210702 Semantic MediaWiki logo with wordmark.png Expert
  • Trainer for Software Architecture and Requirements Engineering
  • PhD Candidate at RWTH Aachen i5
   


BITPlanLogo2012 138x77.png
BITPlan GmbH
BITPlan
  • Inception: 1999


  • Focus on Software-Engineering 

Training, Consulting, Projects …
  • 

Switch from Lotus Notes to 
Semantic Mediawiki in 2013


  • ProfiWiki since 2015
   


BITPlanLogo2012 138x77.png
Agenda
Wikidata-logo-en.svg Wikidata
20210702 Semantic MediaWiki logo with wordmark.png
Octicons-sync.svg Synchronization
   

{

BITPlanLogo2012 138x77.png
SMW and Wikidata storage
  • Both systems use triples: subject, predicate and object
  • It's a girl revisited:
    It's a girl!
The princess of cambridge was born on [[is born::2015-05-03]] 
Her rank inline to the throne is [[throne rank is::4]].
Her mother is [[mother is::Duchess Kate]].
   

{

BITPlanLogo2012 138x77.png
SMW and Wikidata as databases?
Compare to a relational database:    

{

BITPlanLogo2012 138x77.png
SMW relational operations
   

Row and Column definition operations

Caption text
Operation Manually Via Api Via Wikipush
Create create page and use markup to set property edit wikipush, wikiupload, wikirestore
Read visit link of page read html or markup {Done}} wikibackup, wikipush, wikiedit
Update modify page via edit edit wikiedit, wikirestore, wikipush
Delete delete page delete {Done}} wikinuke

Column value operations

Caption text
Operation Manually Via Api Via Wikipush
Create add property value manually on a page ->
Read use ask queries to retrieve property values edit wikiquery
Update modify property value manually on a page ->
Delete remove property value manually on a page

{

BITPlanLogo2012 138x77.png
Modifying a Single triple
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark 
editing 1 pages in family (dry run)
1/1 (100.00%): editing Alexandra_of_Denmark ...👍 |childOf=F74
 |parentOf=F2
 |spouse=Q20875
+|label=Alexandra of Denmark
 }}
 [[Category:frontend]]
wikiedit -t family --template Person --property label --value "Alexandra of Denmark" -p Alexandra_of_Denmark -f
editing 1 pages in family (forced)
1/1 (100.00%): editing Alexandra_of_Denmark ...✅
   

{

BITPlanLogo2012 138x77.png
Example Usecases
  • Royal Family
  • Scholary communication
   

{

BITPlanLogo2012 138x77.png
Example Royal Family
Coat of arms of the United Kingdom (2022, variant 2).svg


  • 0xe4ab021a.png
   

{

BITPlanLogo2012 138x77.png
Sync Royal Family
smwsync -u -t family --context FamilyContext --topic Person -p desc label died died_at --progress
updating cache for FamilyContext:Person from wiki family ...
stored 59 Person items to /Users/wf/.smwsync/family/FamilyContext/Person.json
59 Person items to sync ...
William IV of the United Kingdom→died_at: 100%|█████| 236/236 [01:48<00:00,  2.18it/s]
   

{

BITPlanLogo2012 138x77.png
Example: Scholary communication
0x1a095a5a.png    

{

BITPlanLogo2012 138x77.png
Sync Event Series
https://cr.bitplan.com/index.php/List_of_EventSeries
smwsync -u -t cr --topic EventSeries -p desc title home --progress
updating cache for CrSchema:EventSeries from wiki cr ...
stored 10 EventSeries items to /Users/wf/.smwsync/cr/CrSchema/EventSeries.json
10 EventSeries items to sync ...
SGAI→homepage: 100%|██████████████████████████████████| 30/30 [00:28<00:00,  1.05it/s]
   

{

BITPlanLogo2012 138x77.png
Sync Institutions
smwsync -u -t cr --topic Institution -p desc home --progress
updating cache for CrSchema:Institution from wiki cr ...
stored 4 Institution items to /Users/wf/.smwsync/cr/CrSchema/Institution.json
4 Institution items to sync ...
Universität Stuttgart→homepage: 100%|█████████████████| 12/12 [00:12<00:00,  1.06s/it]
   

{

BITPlanLogo2012 138x77.png
How SMWSync works
  • Self describing Properties
  • MetaModel, SiDIF, LinkML
  • Mapping SMW Properties to Wikidata properties
  • Command Line Tool
   

{

BITPlanLogo2012 138x77.png
Self describing Properties
   

{

BITPlanLogo2012 138x77.png
MetaModel, SiDIF, LinkML
YpgenScreenShotFamilyContext2023-04-19.png
ypgen --wikiId family --context FamilyContext --serve --host localhost

FamilyContext SiDIF is in legacy SiDIF triple format.

pyMetaModel mm_cmd command line tool will generate LinkML yaml markup and other target formats. LinkML can then generate even form format see:

pyMetaModel family example files
   

{

BITPlanLogo2012 138x77.png
Mapping SMW Properties to Wikidata properties ==
# list of property mappings
# for family  versus wikidata
- topic: Person
  prop_list:
  - pid: qid
    smw_prop: qid
  - arg: label
    pid: label
    smw_prop: label
  - arg: desc
    pid: description
    pid_label: description
    smw_prop: description
  - arg: died
    pid: P570
    pid_label: date of death
    smw_prop: died
  - arg: died_at
    pid: P20
    pid_label: place of death
    smw_prop: died_at
   

{

BITPlanLogo2012 138x77.png
Command Line Tool
usage: smwsync [-h] [-a] [--context CONTEXT] [-cpm CREATEPROPERTYMAP] [-d] [--dry]
               [-e ENDPOINT] [--progress] [-p PROPS [PROPS ...]] [--proplist]
               [-pm PROPERTYMAP] [-pk PRIMARYKEY]
               [-pkv PRIMARYKEYVALUES [PRIMARYKEYVALUES ...]] [-t TARGET] [-u]
               [--topic TOPIC] [--verbose] [-V]
   

{

BITPlanLogo2012 138x77.png
Todos
  • synchronization of complete entities or sets of properties (--all option for properties)
    • detection of sync conflicts
  • bidirectional sync
    • publish to wikidata from smw
  • GUI
    • Sync dialog
  • generation of smw topics and properties from wikidata
  • mapping information in property definition page
  • support of intext annotation
   

{

BITPlanLogo2012 138x77.png
Links
   

{

BITPlanLogo2012 138x77.png
Questions / Discussion
Bitplanpartner.png