graph-migration

Migrate Classic Graph to Core Graph

Generate Core Graph Schema

Graph Schema Migration tool creates classic graph frame. Based on classic graph frame schema, the tool generates core graph schema.

There are two types of format when outputting the schema, cql and gremlin. Cql format schema is the CQL script to create graph keyspace and tables under cqlsh. Gremlin format schema is the groovy script to create core graph schema under gremlin-console or Studio.

Run the following command to output the core graph schema generation script when the DSE server is up:

dse graph-migrate-schema [-cql | -gremlin] <classic_graph> <core_graph>

• -cql: specifies the output as cql format.
• -gremlin: specifies the output as gremlin format.
• <classic_graph>: the name of the existing classic graph you would like to migrate from.
• <core_graph>: the name of the new core graph you would like to create.

Please note that the following are NOT supported:

• Meta properties
• Multi properties
• Indexes

All properties are single. Multi properties are migrated as a single property. Meta properties, Indexes, and MVs are dropped. If custom handling of meta and multi properties is needed, users should modify the generated core graph schema accordingly, some properties may need to be renamed or dropped.

To save the generated output as a script into a file, you may run the following:

dse graph-migrate-schema -gremlin reviewerRating nreviewerRating > core_schema.groovy

To create the core graph schema, pass the generated script to gremlin-console or cqlsh depending on the type of format.

Example:

dse gremlin-console -e core_schema.groovy

Migrate Data

The following provided script is just an example of the data migration process.

Please note that if you have modified the generated schema in the previous step, you should modify the migration script accordingly. See "Development notes" for more information.

build:

sbt package

and then submit to Spark:

dse spark-submit target/scala-2.11/graph-migration_2.11-0.1.jar <classic_graph> <core_graph>

Example:

dse spark-submit target/scala-2.11/graph-migration_2.11-0.1.jar reviewerRating nreviewerRating

Development notes

Reference: com.datastax.graph.MigrateData

• migrateVertices()

  This method enumerates core vertex labels, selects proper rows from classic DGF vertices and writes them to appropriate tables.

• migrateEdges()

  This method enumerates core edge labels, extracts (inLabel, edgeLabel, outLabel) triplet from it, selects proper rows from classic DGF edges, converts edge ids into the core format and writes data to the corresponded table.

• handleMultiAndMetaProperies()

  This method is expected to be overridden to properly migrate multi and meta properties. By default, it just drops all metadata and selects the first multi properties.

• getClassicPropertyName()

  This method should be overridden if the property name was manually changed in the schema generated by the graph-migrate-schema.