The purpose of this chapter is to provide some details about the way TheVirtualBrain instance/installation can exchange data with other TheVirtualBrain instances, or with other applications available in the neuroscience community. Currently, there are several applications that can analyze and record brain activity in different formats, and one of the goals of TheVirtualBrain is to allow users from different backgrounds to have quick access to their recorded data.
To achieve this, we have implemented some mechanisms to IMPORT / EXPORT data at different levels and formats:
Warning
During export and import operations TheVirtualBrain does not apply any space transformation, so users have to ensure their data (especially in case of import) is generated/stored in the same space.
TheVirtualBrain‘s default project contains data in a space where the nose is pointing in the direction of -y and the left ear in the direction of +x. The space is right handed: +z points up.
Before proceeding with more details about data exchange, it would be helpful to give you an idea how TheVirtualBrain stores its data. Basically there are two major storage areas:
An important aspect of TheVirtualBrain storage is that each data/datatype has associated a GUID, which makes it unique on every system where that data exists.
TheVirtualBrain product can be installed both on a server, to be used concurrent by multiple users, but also as a standalone application on a desktop/laptop for personal use. Specifically for the second scenario, there was an important request to allow people to exchange data. So, TheVirtualBrain has a mechanism to export and import an entire project to another system.
Using TheVirtualBrain interface, any user can export their projects in a custom format that can be transferred to other users.
Export results are a ZIP file (named: date + project name), containing in a folder structure, all the details about the project. More specifically, it contains:
A root level XML file with details about the project itself
Folders for each operation performed as part of the project
Operation folder contains an XML file with details of the operation and a set of H5/HDF5 files for each data type generated during that operation.
Note
each of the H5 files has a structure as described above in TheVirtualBrain Storage section.
A project exported on one system can now be imported on another machine. In the projects area, TheVirtualBrain offers the possibility to import a project packed as ZIP file.
To import a project, the user has to provide a ZIP with the same structure like the one described above for ‘Export Project’.
Important
The same project cannot be imported multiple times on the same machine, because each project/data has a unique identifier (GUID).
Using TheVirtualBrain interface, users can view all data types associated with a project and choose to export individual pieces of data.
The Export Datatype operation results in a file with a format specific to TheVirtualBrain; it is not a standard format that can be used automatically by other software. This is basically HDF5/H5 format [http://www.hdfgroup.org/HDF5] which, for each data type, contains both data and metadata. These files can be easily opened in Python / Matlab / Java / C++ for additional processing.
In case you want to process HDF5 files with Matlab you can find API documentation here:
Note
The HDF5 functionality referenced above was only introduced in Matlab 2011a.
Note
In the future other data formats might be supported as export format from TVB, but for now, the HDF5 is the only format available at export time.
As a result of a Simulation or Analyze function, TheVirtualBrain can generate either a data type or a group of data types. Each of such structures can be exported as follows:
Probably this is the most important feature of data exchange, since it allows TheVirtualBrain to bring together data generated independently by other systems/applications and allows it’s users to perform different analyses on it and visualize them. Since there is an abundance of formats available for neuroimaging data, TheVirtualBrain tries to support as many as possible for an improved user experience.
Warning
In case the imported data includes/represents a surface, TheVirtualBrain does an extra check regarding the number of vertices of that surface. Basically you can not import into TheVirtualBrain a surface that has more vertices than a MAX value.
This MAX value is defines and can be changed in the Application Settings area, depending on the configuration/performance of your hardware.
In correlation with export operations, TheVirtualBrain interface allows import of data in TheVirtualBrain format that has been exported from other systems. This format applies to any TheVirtualBrain data type. Depending on the uploaded file format, imported data can be as follows:
NIFTI [http://www.nitrc.org/projects/nifti ] is a standard format maintained by “The Neuroimaging Informatics Technology Initiative (NIfTI) and NIfTI Data Format Working Group” and allows the exchange of data with different meanings (imaging data, statistical values, etc.; stored as vectors, matrix, label set or mesh). NIFTI data can be stored in <.nii> or <.hdr+.img> files, or any of these in zipped format (<.gz> files).
For the moment, TheVirtualBrain accommodates import of Volume Time Series from NIFTI files.
For import, TheVirtualBrain users can upload either .nii or .gz files containing NIFTI data in the format specified by [http://www.nitrc.org/projects/nifti]
TheVirtualBrain allows users to import data about sensors used for brain imaging. More specifically, TheVirtualBrain supports three types of sensors: EEG, MEG and INTERNAL. During the import process, the user has to select a file to import and the type of the imported sensors. Based on the selected type, the data from the uploaded file will be processed accordingly.
During import, the user might upload either a TXT file or a zipped TXT in bz2 format. This TXT file should contain data separated by spaces and grouped as follows:
This feature allows import of connectivity from a ZIP file. The ZIP should contain files with connectivity details as follows:
ZIP file should include files with the following naming schema and format:
Using this option, users have the possibility to import a surface from a more human readable format into TVB. Basically users have to upload a zip file containing surface data and specify what type of surface they upload (Cortical Surface, Brain Skull, Skull Skin or Skin Air).
Uploaded ZIP file should contain files with a specified naming schema and format as follow:
There are systems/applications that generate and store surface data in two parts: for left and right side. If this is the case, the imported ZIP file is expected to contain text files with the same naming and format, but the name should contain letter “r” or “l” at the end of the suffix (e.g. <trianglesl.txt> and <trianglesr.txt>)
OBJ is a generic 3d geometry format. Many 3d authoring tools can export geometry in this format.
An overview of the OBJ file format can be found on Wikipedia TVB supports only a subset of the specification. Meaning that only geometry data is considered and accepted forms for faces attributes are: triangles or quads. We ignore at import time features such as texture coordinates, materials and groups.
This is a geometry format (http://www.nitrc.org/projects/gifti/) under the Neuroimaging Informatics Technology Initiative (NIfTI) that allows exchange of brain data (surface, time series, shapes, labels ...). Basically format is XML based which stores both data and associated metadata in a single file, with .gii extension.
If uploaded .gii file contains a surface (Cortical Surface or SkinAir) during import TheVirtualBrain stores found vertices / triangles and computes normals for them.
In case .gii file contains a TimeSeries, user will be asked to specify what is the surface for which TimeSeries is imported. Important to know: number of vertices from imported time series must be the same as the one selected for surface. Otherwise import procedure will fail.
This is a standard format, supported by a large community so all details about it and samples can be found here:
Note
At this moment TheVirtualBrain supports only import of data from a single .gii file. It does not handles cases when metadata is defines in .gii (XML) file and real data in external files.
CFF (Connectome File) is a complex format that tries to put together all data necessary for brain simulations or analysis. Because of its complexity and lack of support from the community, this format is not used very often. For this reason, we decided to implement import only of a custom form of CFF, for demo purposes. Support for CFF import might be removed in the future versions.
The current TheVirtualBrain version includes a set of demo data, housed in a folder that contains two CFF files which could be imported for testing.
Since CFF is a complex format you can use it for uploading single data (e.g one surface, connectivity, local connectivity, region mapping) but also you could group multiple such data into a single CFF file.
For this feature, the user has to upload a CFF file (which is basically a ZIP file) containing a root file <meta.cml> which describes the content of the archive. This file specifies what data types are packed (e.g. connectivity, surface, region mapping) and which files contain data for these types. In our demo data, files are in different formats: starting from raw data (numpy dump), GIFTI, NXGPickle.
A Region Mapping in TheVirtualBrain is a vector, defining a map between a Cortical Surface and a Connectivity. At import time, you will need to have at least 2 entities in TheVirtualBrain system: Connectivity and Cortical Surface. The two entities need to be spatially aligned (overlap correctly in 3D space).
For this upload we expect a text file (possibly compressed with bz2). The text file should have no headers, only numeric values separated with spaces.
The file is expected to hold a vector of length number of vertices in the Cortical Surface. The numeric values should be in the interval (0...n-1), where n is the number of regions in the connectivity.
A Projection Matrix, is intended to define a mapping from a source object and a set of sensors. The source entity can be either a Cortical Surface or a Connectivity, in TheVirtualBrain. In order for this import to work, you will need to have previously imported in TheVirtualBrain: both the source and the sensors entities.
For this upload we expect a single text file, with numeric values, space and line separated. The numeric values in the uploaded file should hold a matrix of size (n, m). n is the number of sensors, and m is the number of nodes. When the projection matrix we want to import is a Surface Projection Matrix, m will be the number of vertices in the target Cortical Surface. When the projection matrix is a region-level one, m will be the number of regions in the Connectivity. Having headers in the text file is not accepted. An incorrect number of values (lines or rows) in the Projection Matrix will also raise an exception.