SimpleGraph

From BITPlan Wiki
Revision as of 16:29, 13 March 2020 by Wf (talk | contribs)
Jump to navigation Jump to search

SimpleGraph

OsProject
id  com.bitplan.simplegraph
state  
owner  BITPlan
title  SimpleGraph System API wrapper
url  https://github.com/BITPlan/com.bitplan.simplegraph
version  0.0.5
description  
date  2019-03-23
since  
until  

is an open source project that allows to wrap Systems APIs in a way that graph algorithms and storage can be applied. As an implementation Apache Tinkerpop/Gremlin is used.


Click here to comment see SimpleGraph

SimpleGraph uses the Hub and Spoke and Adapter patterns heavily.

IntegrationSpaghetti.gif MessageBroker.gif

Modules

There are currently 21 Modules available for SimpleGraph. Each module wraps an "external" API to make the functions and data behind that API available for graph processing with Apache Tinkerpop / Gremlin.

Module Hub and Spoke

Module Details

|- |style="text-align: center; width: 120px" | Circle-icons-calendar.svg CalDAV |style="text-align: left" | CalDAV |style="text-align: left" | ical4j library for parsing and building iCalendar data models |style="text-align: left" | makes Calendar data available via ical4j |-|- |style="text-align: center; width: 120px" | Farm-Fresh vcard.png CardDAV |style="text-align: left" | CardDAV |style="text-align: left" | [ ] |style="text-align: left" | makes VCard data available |-|- |style="text-align: center; width: 120px" | Microsoft Excel 2013-2019 logo.svg Excel |style="text-align: left" | Excel |style="text-align: left" | Apache POI XSSF/HSSF |style="text-align: left" | makes Microsoft Excel workbooks accessible via the Apache POI API |-|- |style="text-align: center; width: 120px" | Folder.svg FileSystem |style="text-align: left" | FileSystem |style="text-align: left" | java.io.File |style="text-align: left" | makes your FileSystem accessible via the Java FileSystem API |-|- |style="text-align: center; width: 120px" | SFA Polygon with hole.svg GeoJSON |style="text-align: left" | GeoJSON |style="text-align: left" | GeoJSON support for Google gson library |style="text-align: left" | makes GeoJSON data available |-|- |style="text-align: center; width: 120px" | Octocat.png GitHub |style="text-align: left" | GitHub |style="text-align: left" | GitHub GraphQL Api v4 |style="text-align: left" | makes GitHub content accessible to Graph processing. |-|- |style="text-align: center; width: 120px" | HTML5 logo and wordmark.svg HTML |style="text-align: left" | HTML |style="text-align: left" | HTML Cleaner |style="text-align: left" | makes HTML files accessible via HTML Cleaner parser |-|- |style="text-align: center; width: 120px" | JSON vector logo.svg JSON |style="text-align: left" | JSON |style="text-align: left" | JSON |style="text-align: left" | makes JSON parse results accessible to Graph processing. |-|- |style="text-align: center; width: 120px" | Java-Logo.svg Java |style="text-align: left" | Java |style="text-align: left" | javaparser |style="text-align: left" | makes Java code parse results accessible to Graph processing. |-|- |style="text-align: center; width: 120px" | Email Icon.svg Mail |style="text-align: left" | Mail |style="text-align: left" | E-Mail access for rfc822 and MIME formatted Mailbox files (e.g. Thunderbird) |style="text-align: left" | makes Mail data available via Apache Mime4J |-|- |style="text-align: center; width: 120px" | Map.png MapSystem |style="text-align: left" | MapSystem |style="text-align: left" | java.api.Map |style="text-align: left" | supplies a simple wrapper for a graph with nodes that have key/value pairs in form of HashMaps. We would not really need this since Apache Tinkerpop/Gremlin already supplies us with properties per node/vertex. Still this system is useful as a helper system and to illustrate the wrapping concepts and possibilities of SimpleGraph |-|- |style="text-align: center; width: 120px" | Mediawiki logo reworked.svg MediaWiki |style="text-align: left" | MediaWiki |style="text-align: left" | MediaWiki API |style="text-align: left" | makes MediaWiki site content accessible to Graph processing. It exposes the MediaWiki API using the mediawiki-japi Library by BITPlan. |-|- |style="text-align: center; width: 120px" | Pdf by mimooh.svg PDF |style="text-align: left" | PDF |style="text-align: left" | Apache PDFBox |style="text-align: left" | makes Portable Document Format (PDF) files accessible via the Apache PDFBox® API |-|- |style="text-align: center; width: 120px" | Microsoft PowerPoint 2013-2019 logo.svg PowerPoint |style="text-align: left" | PowerPoint |style="text-align: left" | Apache POI XSLF/HSLF |style="text-align: left" | makes Microsoft PowerPoint presentations accessible via the Apache POI API |-|- |style="text-align: center; width: 120px" | SemanticMediaWiki Logo.png SMW |style="text-align: left" | SMW |style="text-align: left" | SemanticMedia Wiki API |style="text-align: left" | makes Semantic MediaWiki accessible via the SMW API |-|- |style="text-align: center; width: 120px" | Snmp.png SNMP |style="text-align: left" | SNMP |style="text-align: left" | SNMP4J Simple Network Monitoring Protocol SNMP Java API |style="text-align: left" | makes Simple Network Monitoring Protocol accessible via SNMP4J |-|- |style="text-align: center; width: 120px" | Database.svg SQL |style="text-align: left" | SQL |style="text-align: left" | Java Database Connectivity (JDBC) API |style="text-align: left" | makes relational SQL databases accessible via the Java JDBC API. |-|- |style="text-align: center; width: 120px" | TripleStore-Icon.png TripleStore |style="text-align: left" | TripleStore |style="text-align: left" | SiDIF-TripleStore |style="text-align: left" | makes BITPlan's SiDIF educational TripleStore accessible |-|- |style="text-align: center; width: 120px" | Wikidata-logo-en.svg WikiData |style="text-align: left" | WikiData |style="text-align: left" | WikiData Toolkit |style="text-align: left" | makes WikiData data available via the Wikidata-Toolki API |-|- |style="text-align: center; width: 120px" | Microsoft Word 2013-2019 logo.svg Word |style="text-align: left" | Word |style="text-align: left" | Apache POI XWPF/HWPF |style="text-align: left" | makes Microsoft Word Documents accessible via the Apache POI API |-|- |style="text-align: center; width: 120px" | Xml logo.svg XML |style="text-align: left" | XML |style="text-align: left" | org.w3c.dom |style="text-align: left" | makes XML dom parse results accessible to Graph processing. |-


Introduction

Motivation

Solving IT Problems across System boundaries can get very difficult. There may be a wealth of APIs which first looks helpful but given the diversity of approaches it a daunting task will need to be tackled to get reasonable results. More often than not projects are not even started since the cost/benefit ratio is not good enough. SimpleGraph aims to supply a unified graph API access to Systems for which this makes sense. In fact for most systems it makes sense to have a graph API. Quite a few problems will get much easier to solve if the subdivision of the problem is done with the goal to apply graph algorithms. See What happens if you see the world as a graph?


Use Cases

Mix and Match Office and other data

Let's assume we have a business that works in the following manner:

  1. There are pricelist for product categories in Microsoft Excel files
  2. Product specifications are in PDF Format - the files are referenced in the Excel files
  3. Invoices are written in Microsoft Word
  4. There is an address book of customers in VCard format
  5. Orders are handled via e-mail (that is automatically produced by the companies shop website)
  6. Monthly reports are created with PowerPoint

The monthly reports should in the future be modified to give geographical information. Basically a revenue/region sales count per product/region information is wanted.

The 1000 dollar question - can this report be created efficiently semi-manually with reasonable software effort? Sadly it sounds more like 10.000 or 100.000 dollar question.

There are at least 6 different systems involved. SimpleGraph simplifies accessing all 6 systems.


Examples

Air Routes

The AirRoutes example is taken from Practical Gremlin: An Apache TinkerPop Tutorial by Kelvin Lawrence see also https://github.com/krlawrence/graph.

JUnit Test case

We'd like to read in the air-routes graph described above and create an Excel Work book from it. see TestExcelSystem.java

Java Source Code

  ExcelSystem es = new ExcelSystem();
  Graph graph = TestTinkerPop3.getAirRoutes();
  GraphTraversalSource g = graph.traversal();
  Workbook wb = es.createWorkBook(g);
  assertEquals(6, wb.getNumberOfSheets());
  es.save(wb, testAirRouteFileName);

View the resulting File:Air-routes.xlsx Excel file to see what the tabular version of the graph looks like.

Links

Documentation

Links