3dCompare

Scientists often seek to compare diverse data types to gain insights into variations in structure and dynamics. However, most software environments focus on a single data type, which limits comparisons across data types, for example data volumes with point clouds. Virtual environments can facilitate these comparisons by providing the ability to render different objects in the same space, allowing users to manipulate the objects in ways appropriate to the data types, and providing tools to evaluate dynamics.

KeckCAVES researchers are approaching developing such a system using a python interface to VRUI, called VROOM as the basis for 3dCompare. The python programming language is object oriented and provides data structures that are stored in memory and manipulated in ways that are natural to the specific data type. These objects are then translated and transferred to VRUI for rendering and interaction using vroom to translate graphics and pyvrui to translate interactions. The long-term goal is to expose the full VRUI interactive and quantitative functions to objects created in python.

The CI-TEAM project supported the development of vroom and PyVrui, which currently allow interactive manipulation of diverse virtual objects to:

1. Provide access to multiple objects indexed by user-supplied metadata;
2. Allow the user to manipulate objects independently, including non-linear deformations, rigid motions and scaling;
3. Implement intuitive virtual physics (e.g. forbidding object interpenetration, simulating gravity, etc.) or, alternatively, allow virtual interactions that are impossible in the physical world (e.g. overlap two objects to compare their geometry);

Object types that can be incorporated and compared include point clouds, texture-mapped geometrical objects, including wireframe objects, and text labels. Dynamics for objects can be programmed in python with the user program controlling all aspects of object behavior (in the absence of real-time interaction through the VRUI interface.) The next steps in development will focus on implementing deeper interactivity:

1. Incorporate appropriate tools for quantitative comparison of objects – for example, allowing the user to add point correspondences to objects, with real-time feedback on local or global quality of the implied object correspondences;
2. Allow objects to be connected to each other (e.g. with rods or springs) to study behaviors or to allow users to assemble composite objects from fragments (e.g. bones in a skeleton, shards of pottery), and provide automatic tools for optimizing assemblies;
3. Enable spatial arrangements for annotating and sharing comparisons with collaborators and for display.

This development effort has included researchers and students from Computer Science, Geology, Physics, Anthropology, Science & Technology Studies, and English.

The framework for 3dCompare has been established but implementing the full interactivity of VRUI remains challenging. Thus, 3dCompare is now capable of object display and manipulation but not yet quantitative comparisons of objects. It has been used extensively for outreach and art projects, specifically the Dream Vortex and “Outside the Vortex”

The Dream Vortex is a collaboration between Prof. Dawn Sumner (UCD), Prof. Meredith Tromble (San Francisco Art Institute) and Complexity Sciences Center (UC Davis) members to interactively and immersively explore 3D networks of dreams. The goal of the project is to combine social aspects of being a KeckCAVES researcher with the technology through the artistic medium of hand drawn images. Tromble collected dreams volunteered by KeckCAVES researchers, represented them in drawings, and conceptually connected them through network properties. Sumner transformed the drawings into dynamic objects in the 3dCompare project landscape that can be manipulated by users using a 3d headset and interaction device. A mini-documentary of the project is available here. The project has been shown at various venues:

• 3/4/2013: Art, Technology, and Culture Presentation, UC Berkeley;
• 4/25/2013: Art Practice Seminar, UC Davis
• 7/2014: Scientific Delirium Madness, Djerassi Resident Artist Program and Leonardo/The Society for the Arts, Sciences, and Technology – Fellowship to participate in a four-week residency to continue development of the Dream Vortex.
• 11/8-11/2015: Alliance for the Arts in Research Universities 2015 National Meeting “Ground Works: Improving and Supporting Practice in the Third Space”. Dream Vortex was selected as one of 6 Transdisciplinary Exemplars for presentation, critique, and review.

“Outside the Vortex” is an ongoing collaboration among KeckCAVES geologist Prof. Dawn Sumner (UCD), artist Prof. Meredith Tromble (San Francisco Art Institute), choreographer Donna Sternberg, and KeckCAVES physicist James Crutchfield (UCD) exploring exclusion and difference in STEM fields and their roles in discovery. Tromble collected stories of exclusion and difference experienced by STEM researchers, and represented them in suites of drawings and narrative stories.

Crutchfield used synthesized voices to provide the narrative, and Sumner programmed the drawings for dynamic projection onto stage scrims. Sternberg choreographed movements that expressed the action, effects and emotions of exclusion as well as discovery. An initial workshop performance was staged by Donna Sternberg and Dancers June 24 and 25, 2016 at Diavolo Studio, Los Angeles. (Video)