Difference between revisions of "Gremlin"

From BITPlan Wiki
Jump to navigation Jump to search
Line 98: Line 98:
 
All steps are based on five {{Link|target=Gremlin#General_Steps|title=general steps}}.
 
All steps are based on five {{Link|target=Gremlin#General_Steps|title=general steps}}.
 
Click on any of the steps below to see the explanation for the step
 
Click on any of the steps below to see the explanation for the step
−
<graphviz>
+
<source lang='bash' lines>
 
digraph steps {
 
digraph steps {
 
   rankdir="LR";
 
   rankdir="LR";
Line 195: Line 195:
 
   }
 
   }
 
}
 
}
−
</graphviz>
+
</source>
  
 
== map Steps ==
 
== map Steps ==

Revision as of 19:10, 25 April 2019

Gremlin programming language.png

Gremlin is the graph traversal language of Apache TinkerPop. Gremlin is a functional, data-flow language that enables users to succinctly express complex traversals on (or queries of) their application's property graph. Every Gremlin traversal is composed of a sequence of (potentially nested) steps. A step performs an atomic operation on the data stream. Every step is either a map-step (transforming the objects in the stream), a filter-step (removing objects from the stream), or a sideEffect-step (computing statistics about the stream). The Gremlin step library extends on these 3-fundamental operations to provide users a rich collection of steps that they can compose in order to ask any conceivable question they may have of their data for Gremlin is Turing Complete.

Explaining Gremlin

There are different levels on which gremlin can be explained:

  1. Mathematical background as explained in Marko Rodriguez's paper The Gremlin Graph Traversal Machine and Language
  2. Generic API as explained in the Tinkerpop documentation
  3. Specific API (Java) as explained in the Javadocs page
  4. Specific "modern" Example mostly used for tests and explanations regarding Gremlin

On this page the goal is to cover all 4 levels with a focus on Java being applied to the modern example. The source code TestSteps.java is available on github.

Graph

A Graph G= (V, E) consist of a finite set of vertices V and a finite set of edges E ⊆ V×V.

An Element of a Graph is either a vertice or an edge.

A Propertygraph allows all elements (vertice or edge) of a graph to have properties. Each property is a name/value pair.

The Modern example

The "modern" graph is shipped with gremlin as a standard example. tinkerpop-modern.png

The graph has 6 edges and 6 vertices.

It consists of :

  1. vertice person (name: marko, age:29)
  2. vertice person (name: vadas, age:27)
  3. vertice software (name: lop, lang: java)
  4. vertice person (name: josh, age:32)
  5. vertice software (name: ripple, lang: java)
  6. vertice person (name: peter, age:35)
  7. edge knows 1->2 (weight: 0.5)
  8. edge knows 1->4 (weight: 1.0)
  9. edge created 1->3 (weight: 0.4)
  10. edge created 4->5 (weight: 1.0)
  11. edge created 4->3 (weight: 0.4)
  12. edge created 6->3 (weight: 0.2)

In Gremlin edges and vertices have a set of properties. Each property is a name/value pair. One important property is the id of a vertice or edge. E.g. the vertice for peter has the id 6 and a property with the name "age" and the value 35 and another property with the name "name" and the value "peter".

GraphTraversal

One of the core concepts of tinkerpop/gremlin is the GraphTraversal It's interface has a generic definition as: 

public interface GraphTraversal<S,E> extends Traversal<S,E>

and at https://markorodriguez.com/ the Author Marko Rodriguez explains the ideas behind using an generic approach vor handling Graphs. The Java implementation is available on github.

S is a generic Start class, and E is a generic End class as explained in the Apache Tinkerpop documentation.

GraphTraversalSource

A Graph Traversal Source is the starting point for working with a graph. The convention is to name this starting point 

g

or 

g()

In our tests we'll use a GraphTraversalSource for the modern example 

  /**
   * common access to GraphTraversalSource
   * @return - the graph traversal
   */
  public GraphTraversalSource g() {
    Graph graph = TinkerFactory.createModern();
    GraphTraversalSource g = graph.traversal();
    return g;
  }


JUnit Testcase

  @Test
  public void testTraversal() {
    assertEquals(6,g().E().count().next().longValue());
    assertEquals(6,g().V().count().next().longValue());
  }

E() gives you access to the edges of a graph traversal. V() gives you access to the vertices of a graph traversal. In the above example we simply count the edges and vertices and check our assumption that there are 6 edges and 6 vertices in the modern example graph.

Steps

As explained in Gremlin_Basics: "The Gremlin graph traversal language defines approximately 30 steps which can be understood as the instruction set of the Gremlin traversal machine.

A regular computer has a CPU with an Instruction Pointer which tells the machine to take the instruction at that memory address and execute it next. There are also instruction that can manipulate the instruction pointer with the effect of the return from a function or a goto to a different part of the program.

Gremlin instead works on a sequence of steps and each step the "graph traversal machine" will take it's current state and execute the step to reach a new state of affairs.

The gremlin steps are useful in practice, with typically only 10 or so of them being applied in the majority of cases. Each of the provided steps can be understood as being a specification of one of the 5 general types enumerated below". step-types.png

Stephierarchy

All steps are based on five general steps. Click on any of the steps below to see the explanation for the step 

digraph steps {
  rankdir="LR";
  step [ URL="[[Gremlin#Steps|steps]]" ] 
  subgraph cluster_general {
    label="general steps";
    graph[style=dotted];
    map [ URL="[[Gremlin#map Step|map]]" ] 
    flatMap [ URL="[[Gremlin#flatMap Step|flatMap]]" ] 
    filter [ URL="[[Gremlin#filter Step|filter]]" ] 
    sideEffect [ URL="[[Gremlin#sideEffect Step|sideEffect]]" ] 
    branch [ URL="[[Gremlin#branch Step|branch]]" ] 
  
    map -> step
    flatMap -> step
    filter -> step
    sideEffect -> step
    branch ->step 
  }
  subgraph cluster_map {
    label="map steps";
    graph[style=dotted];
    id [ URL="[[Gremlin#id Step|id]]" ] 
    label [ label="label" URL="[[Gremlin#label Step|label]]" ] 
    match [ URL="[[Gremlin#match Step|match]]" ] 
 
    path [ URL="[[Gremlin#path Step|path]]" ] 
    select [ URL="[[Gremlin#select Step|select]]" ] 
    order [ URL="[[Gremlin#order Step|order]]" ] 

    id->map
    label->map
    match->map
    path->map
    select->map
    order->map
  }
  
  subgraph cluster_flatMap {
    label="flatMap steps";
    graph[style=dotted];

    in [ URL="[[Gremlin#in Step|in]]" ] 
    out [ URL="[[Gremlin#out Step|out]]" ] 
    both [ URL="[[Gremlin#both Step|both]]" 

    inE [ URL="[[Gremlin#inE Step|inE]]" ] 
    outE [ URL="[[Gremlin#outE Step|outE]]" ] 
    bothE [ URL="[[Gremlin#bothE Step|bothE]]" ] 

    inV [ URL="[[Gremlin#inV Step|inV]]" ] 
    outV [ URL="[[Gremlin#outV Step|outV]]" ] 
    bothV [ URL="[[Gremlin#bothV Step|bothV]]" ] 

    in -> flatMap
    out -> flatMap
    both -> flatMap

    inE -> flatMap
    outE -> flatMap
    bothE -> flatMap

    inV -> flatMap
    outV -> flatMap
    bothV -> flatMap

    coalesce -> flatMap
  }
 
  subgraph cluster_filter{
    label="filter steps";
    graph[style=dotted];
    and -> filter
    coin -> filter
    has -> filter
    is -> filter
    or -> filter
    where -> filter
  }
  subgraph cluster_sideEffect {
    label="sideEffect steps";
    graph[style=dotted];
    aggregate -> sideEffect
    inject -> sideEffect
    profile -> sideEffect
    property -> sideEffect
    sg [ label="subgraph" ]
    sg -> sideEffect
  }
  subgraph cluster_branch {
    label="branch steps";
    graph[style=dotted];
    choose -> branch
    repeat -> branch
    union -> branch
  }
}

map Steps

id Step

The id step maps the traversal to the ids of the current elements. 

  @Test
  public void testId() {
    List<Object> vids = g().V().id().toList();
    assertEquals(6,vids.size());
    assertEquals("[1, 2, 3, 4, 5, 6]",vids.toString());
    List<Object> eids = g().E().id().toList();
    assertEquals(6,eids.size());
    assertEquals("[7, 8, 9, 10, 11, 12]",eids.toString());
  }

label Step

The label step maps the traversal to the labels of the current elements. 

  @Test
  public void testLabel() {
    List<String> vlabels = g().V().label().toList();
    assertEquals(6,vlabels.size());
    assertEquals("[person, person, software, person, software, person]",vlabels.toString());
    List<String> elabels = g().E().label().toList();
    assertEquals(6,elabels.size());
    assertEquals("[knows, knows, created, created, created, created]",elabels.toString());
  }

match Step

see https://stackoverflow.com/questions/55609832/is-threre-a-document-about-how-gremlin-match-works

path Step

select Step

order Step

flatMap Steps

in Step

The in step maps the current elements to the vertices at the end of the ingoing edges. 

  @Test
  public void testIn() {
    assertEquals("[v[1], v[1], v[4], v[6], v[1], v[4]]",
        g().V().in().toList().toString());
    assertEquals("[v[1], v[4], v[6], v[4]]",
        g().V().in("created").toList().toString());
    assertEquals("[v[1], v[1]]", g().V().in("knows").toList().toString());
    assertEquals("[v[1], v[1], v[4], v[6], v[1], v[4]]",
        g().V().in("created","knows").toList().toString());
  }

out Step

The out step maps the current elements to the vertices at the end of the outgoing edges. 

  @Test
  public void testOut() {
    assertEquals("[v[3], v[2], v[4], v[5], v[3], v[3]]",
        g().V().out().toList().toString());
    assertEquals("[v[3], v[5], v[3], v[3]]",
        g().V().out("created").toList().toString());
    assertEquals("[v[2], v[4]]", g().V().out("knows").toList().toString());
    assertEquals("[v[3], v[2], v[4], v[5], v[3], v[3]]",
        g().V().out("created","knows").toList().toString());
  }

both Step

The both step maps the current elements to the vertices at the boths ends of the edges. 

  @Test
  public void testBoth() {
    assertEquals("[v[5], v[3], v[1]]",
      g().V(4).both().toList().toString());
    assertEquals("[v[5], v[3]]",
        g().V(4).both("created").toList().toString());
    assertEquals("[v[1]]", g().V(4).both("knows").toList().toString());
    assertEquals("[v[5], v[3], v[1]]",
        g().V(4).both("created","knows").toList().toString());
  }

General Steps

filter Step

Continues processing based on the given filter condition. 

  @Test
  public void testFilter() {
    assertEquals(3,g().V().filter(out()).count().next().longValue());
    assertEquals(4,g().V().filter(in()).count().next().longValue());
    assertEquals(5,g().E().filter(values("weight").
      is(P.gte(0.4))).count().next().longValue());
  }

There are 3 vertices having outgoing edges and 4 vertices having incoming edges in the modern example graph. There are 4 edges having a weight>=0.4;

map Step

A map step transforms the current step element to a new element (which may be empty). see also https://stackoverflow.com/questions/51015636/in-gremlin-how-does-map-really-work 

 @Test
  public void testMap() {
    assertEquals(6,g().V().map(values("name")).count().next().longValue());
    assertEquals(4,g().V().map(hasLabel("person")).count().next().longValue());
    assertEquals(2,g().V().map(has("lang","java")).count().next().longValue());
    List<Edge> outEdges = g().V().map(outE()).toList();
    assertEquals(3,outEdges.size());
    List<Object> edges = g().E().map(has("weight",0.4)).toList();
    assertEquals(2,edges.size());
    for (Object edge:edges) {
      assertTrue(edge instanceof Edge);
    }
  }

There are 6 vertices having a name property. There are 4 vertices with a "person" label. There are 2 vertices with the lang property having the value "java".There are 3 vertices having out edges. The toList() call returns a list of Edges. There are 2 edges having a weight of 0.4. The map step toList() returns a list of the edges for this last example (which are returned as generic objects).

flatMap Step

A flatMap step transforms the current step in a one to many fashion. 

 @Test
  public void testflatMap() {
    assertEquals(6,g().V().flatMap(values("name")).count().next().longValue());
    assertEquals(4,g().V().flatMap(hasLabel("person")).count().next().longValue());
    assertEquals(2,g().V().flatMap(has("lang","java")).count().next().longValue());
    List<Edge> outEdges = g().V().flatMap(outE()).toList();
    assertEquals(6,outEdges.size());
    List<Object> edges = g().E().flatMap(has("weight",0.4)).toList();
    assertEquals(2,edges.size());
    for (Object edge:edges) {
      assertTrue(edge instanceof Edge);
    }
  }

Note the difference to the testMap step. Only the outE() parameter behaves different. In the map() case only the first Edge is considered - in the flatMap case all edges are considered.

sideEffect Step

A sideEffect steps performs some operation on the traverser and passes it to the next step. 


  @Test
  public void testSideEffect() {
    assertEquals(6,g().V().sideEffect(addE("sideedge")).outE().
      hasLabel("sideedge").count().next().longValue());
  }

The sideffect in this example JUnit test case adds edges "on the fly".

branch Step

Split the traverser 

  @Test
  public void testBranch() {
   
  }

What links here

Links

Stackoverflow Questions

Recipes

Practical Gremlin: An Apache TinkerPop Tutorial by Kelvin Lawrence

load PDF

Traversing Graphs with Gremlin

Retrieved from "https://wiki.bitplan.com/index.php?title=Gremlin&oldid=8808"