Navigational Data Management: a general solution to storage, exploration, and annotation of relationships in scientific data sets

The size and complexity of scientific research data sets is growing very rapidly. Many projects are long running collaborations between evolving groups of researchers. Integrating and documenting these large heterogeneous data collections is needed for these data to be exploited, and to retain their value and meaning over time. Most existing projects have application specific solutions for these data management problems. The Navigational Data Management (NDM) project provides a general solution that can be applied irrespective of the science domain. The system represents data and annotations as generalized objects. It stores relationships as labeled property graphs. URIs provide a mechanism to attach external data objects: files, MDSplus records, drawings, etc... NDM stores the metadata and data schemas, and application behaviors as first class objects. These can then be modified, added to, and even annotated as the needs of the research group evolve.

This new tool set is under construction and being applied to some initial projects at several institutions. It is a single page application (SPA) using modern javascript front-end tools, and a graph database as the back-end.