mdf3reader module documentation

Measured Data Format file reader module for version 3.x

Platform and python version

With Unix and Windows for python 2.6+ and 3.2+

Author:Aymeric Rateau

Created on Sun Oct 10 12:57:28 2010

Dependencies

Attributes

PythonVersion : float
Python version currently running, needed for compatibility of both python 2.6+ and 3.2+

mdf3reader module

class mdfreader.mdf3reader.DATA(fid, pointer)

Bases: dict

DATA class is organizing record classes itself made of channel. This class inherits from dict. Keys are corresponding to channel group recordID. A DATA class corresponds to a data block, a dict of record classes (one per channel group). Each record class contains a list of channel class representing the structure of channel record.

Attributes

fid (io.open) file identifier
pointerToData (int) position of Data block in mdf file
BlockLength (int) total size of data block

Methods

addRecord(record) Adds a new record in DATA class dict
read(channelSet) Reads data block
loadSorted(record, nameList=None) Reads sorted data block from record definition
loadUnSorted(nameList=None) Reads unsorted data block, not yet implemented
addRecord(record)

Adds a new record in DATA class dict

Parameters:

record class

channel group definition listing record channel classes

loadSorted(record, nameList=None)

Reads sorted data block from record definition

Parameters:

record class

channel group definition listing record channel classes

channelSet : set of str, optional

list of channel names

Returns:

numpy recarray of data

loadUnSorted(nameList=None)

Reads unsorted data block from record definition

Parameters:

record class

channel group definition listing record channel classes

channelSet : set of str, optional

list of channel names

Returns:

numpy recarray of data

read(channelSet, filename)

Reads data block

Parameters:

channelSet : set of str, optional

list of channel names

filename : str

name of file

mdfreader.mdf3reader.expConv(data, conv)

apply exponential conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

mdfreader.mdf3reader.formulaConv(data, conv)

apply formula conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

Notes

Requires sympy module

mdfreader.mdf3reader.linearConv(data, conv)

apply linear conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

mdfreader.mdf3reader.logConv(data, conv)

apply logarithmic conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

class mdfreader.mdf3reader.mdf3(fileName=None, channelList=None, convertAfterRead=True, filterChannelNames=False, noDataLoading=False, compression=False, convertTables=False, metadata=2)

Bases: mdfreader.mdf.mdf_skeleton

mdf file version 3.0 to 3.3 class

Attributes

fileName (str) file name
MDFVersionNumber (int) mdf file version number
masterChannelList (dict) Represents data structure: a key per master channel with corresponding value containing a list of channels One key or master channel represents then a data group having same sampling interval.
multiProc (bool) Flag to request channel conversion multi processed for performance improvement. One thread per data group.
convertAfterRead (bool) flag to convert raw data to physical just after read
filterChannelNames (bool) flag to filter long channel names from its module names separated by ‘.’
file_metadata (dict) file metadata with minimum keys: author, organisation, project, subject, comment, time, date

Methods

read3( fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True) Reads mdf 3.x file data and stores it in dict
_getChannelData3(channelName) Returns channel numpy array
_convertChannel3(channelName) converts specific channel from raw to physical data according to CCBlock information
_convertAllChannel3() Converts all channels from raw data to converted data according to CCBlock information
write3(fileName=None) Writes simple mdf 3.3 file
read3(fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True, filterChannelNames=False, compression=False, metadata=2)

Reads mdf 3.x file data and stores it in dict

Parameters:

fileName : str, optional

file name

info : mdfinfo3.info3 class

info3 class containing all MDF Blocks

multiProc : bool

flag to activate multiprocessing of channel data conversion

channelList : list of str, optional

list of channel names to be read If you use channelList, reading might be much slower but it will save you memory. Can be used to read big files

convertAfterRead : bool, optional

flag to convert channel after read, True by default If you use convertAfterRead by setting it to false, all data from channels will be kept raw, no conversion applied. If many float are stored in file, you can gain from 3 to 4 times memory footprint To calculate value from channel, you can then use method .getChannelData()

filterChannelNames : bool, optional

flag to filter long channel names from its module names separated by ‘.’

compression : bool, optional

flag to activate data compression with blosc

metadata: int, optional, default = 2

Reading metadata has impact on performance, especially for mdf 4.x using xml. 2: minimal metadata reading (mostly channel blocks) 1: used for noDataLoading 0: all metadata reading

write3(fileName=None)

Writes simple mdf 3.3 file

Parameters:

fileName : str, optional

Name of file If file name is not input, written file name will be the one read with appended ‘_new’ string before extension

Notes

All channels will be converted to physical data, so size might be bigger than original file

mdfreader.mdf3reader.polyConv(data, conv)

apply polynomial conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

mdfreader.mdf3reader.rationalConv(data, conv)

apply rational conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

class mdfreader.mdf3reader.record(dataGroup, channelGroup)

Bases: list

record class lists Channel classes,
it is representing a channel group

Attributes

CGrecordLength (int) length of record from channel group block information in Byte
recordLength (int) length of record from channels information in Byte
numberOfRecords (int) number of records in data block
recordID (int) recordID corresponding to channel group
recordIDnumber (int) size of recordID
dataGroup (int:) data group number
channelGroup (int) channel group number
numpyDataRecordFormat (list) list of numpy (dtype) for each channel
dataRecordName (list) list of channel names used for recarray attribute definition
master (dict) define name and number of master channel
recordToChannelMatching (dict) helps to identify nested bits in byte
channelNames (set) channel names to be stored, useful for low memory consumption but slow
hiddenBytes (Bool, False by default) flag in case of non declared channels in record
byte_aligned (Bool, True by default) flag for byte aligned record

Methods

addChannel(info, channelNumber)  
loadInfo(info)  
readSortedRecord(fid, pointer, channelSet=None)  
readRecordBuf(buf, channelSet=None)  
readRecordBits(bita, channelSet=None)  
addChannel(info, channelNumber)

add a channel in class

Parameters:

info : mdfinfo3.info3 class

channelNumber : int

channel number in mdfinfo3.info3 class

loadInfo(info)

gathers records related from info class

Parameters:info : mdfinfo3.info3 class
readRecordBits(bita, channelSet=None)

read stream of record bits by bits in case of not aligned or hidden bytes

Parameters:

buf : stream

stream of bytes read in file

channelSet : Set of str, optional

list of channel to read

Returns:

rec : dict

returns dictionary of channel with its corresponding values

readRecordBuf(buf, channelSet=None)

read stream of record bytes

Parameters:

buf : stream

stream of bytes read in file

channelSet : Set of str, optional

list of channel to read

Returns:

rec : dict

returns dictionary of channel with its corresponding values

readSortedRecord(fid, pointer, channelSet=None)

reads record, only one channel group per datagroup

Parameters:

fid : float

file identifier

pointer

position in file of data block beginning

channelSet : Set of str, optional

list of channel to read

Returns:

rec : numpy recarray

contains a matrix of raw data in a recarray (attributes corresponding to channel name)

Notes

If channelSet is None, read data using numpy.core.records.fromfile that is rather quick. However, in case of large file, you can use channelSet to load only interesting channels or only one channel on demand, but be aware it might be much slower.

mdfreader.mdf3reader.tabConv(data, conv)

apply Tabular conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

mdfreader.mdf3reader.tabInterpConv(data, conv)

apply Tabular interpolation conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

mdfreader.mdf3reader.textRangeTableConv(data, conv)

apply text range table conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value

mdfreader.mdf3reader.textTableConv(data, conv)

apply text table conversion to data

Parameters:

data : numpy 1D array

raw data to be converted to physical value

conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict

Returns:

converted data to physical value