mdf3reader module documentation¶
Measured Data Format file reader module for version 3.x
Platform and python version¶
With Unix and Windows for python 2.6+ and 3.2+
Author: | Aymeric Rateau |
---|
Created on Sun Oct 10 12:57:28 2010
Dependencies¶
- Python >2.6, >3.2 <http://www.python.org>
- Numpy >1.6 <http://numpy.scipy.org>
- Sympy to convert channels with formula
Attributes¶
- PythonVersion : float
- Python version currently running, needed for compatibility of both python 2.6+ and 3.2+
mdf3reader module¶
-
class
mdfreader.mdf3reader.
DATA
(fid, pointer)¶ Bases:
dict
DATA class is organizing record classes itself made of channel. This class inherits from dict. Keys are corresponding to channel group recordID. A DATA class corresponds to a data block, a dict of record classes (one per channel group). Each record class contains a list of channel class representing the structure of channel record.
Attributes
fid (io.open) file identifier pointerToData (int) position of Data block in mdf file BlockLength (int) total size of data block Methods
addRecord(record) Adds a new record in DATA class dict read(channelSet) Reads data block loadSorted(record, nameList=None) Reads sorted data block from record definition loadUnSorted(nameList=None) Reads unsorted data block, not yet implemented -
addRecord
(record)¶ Adds a new record in DATA class dict
Parameters: record class
channel group definition listing record channel classes
-
loadSorted
(record, nameList=None)¶ Reads sorted data block from record definition
Parameters: record class
channel group definition listing record channel classes
channelSet : set of str, optional
list of channel names
Returns: numpy recarray of data
-
loadUnSorted
(nameList=None)¶ Reads unsorted data block from record definition
Parameters: record class
channel group definition listing record channel classes
channelSet : set of str, optional
list of channel names
Returns: numpy recarray of data
-
read
(channelSet, filename)¶ Reads data block
Parameters: channelSet : set of str, optional
list of channel names
filename : str
name of file
-
-
mdfreader.mdf3reader.
expConv
(data, conv)¶ apply exponential conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
mdfreader.mdf3reader.
formulaConv
(data, conv)¶ apply formula conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
Notes
Requires sympy module
-
mdfreader.mdf3reader.
linearConv
(data, conv)¶ apply linear conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
mdfreader.mdf3reader.
logConv
(data, conv)¶ apply logarithmic conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
class
mdfreader.mdf3reader.
mdf3
(fileName=None, channelList=None, convertAfterRead=True, filterChannelNames=False, noDataLoading=False, compression=False, convertTables=False, metadata=2)¶ Bases:
mdfreader.mdf.mdf_skeleton
mdf file version 3.0 to 3.3 class
Attributes
fileName (str) file name MDFVersionNumber (int) mdf file version number masterChannelList (dict) Represents data structure: a key per master channel with corresponding value containing a list of channels One key or master channel represents then a data group having same sampling interval. multiProc (bool) Flag to request channel conversion multi processed for performance improvement. One thread per data group. convertAfterRead (bool) flag to convert raw data to physical just after read filterChannelNames (bool) flag to filter long channel names from its module names separated by ‘.’ file_metadata (dict) file metadata with minimum keys: author, organisation, project, subject, comment, time, date Methods
read3( fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True) Reads mdf 3.x file data and stores it in dict _getChannelData3(channelName) Returns channel numpy array _convertChannel3(channelName) converts specific channel from raw to physical data according to CCBlock information _convertAllChannel3() Converts all channels from raw data to converted data according to CCBlock information write3(fileName=None) Writes simple mdf 3.3 file -
read3
(fileName=None, info=None, multiProc=False, channelList=None, convertAfterRead=True, filterChannelNames=False, compression=False, metadata=2)¶ Reads mdf 3.x file data and stores it in dict
Parameters: fileName : str, optional
file name
info : mdfinfo3.info3 class
info3 class containing all MDF Blocks
multiProc : bool
flag to activate multiprocessing of channel data conversion
channelList : list of str, optional
list of channel names to be read If you use channelList, reading might be much slower but it will save you memory. Can be used to read big files
convertAfterRead : bool, optional
flag to convert channel after read, True by default If you use convertAfterRead by setting it to false, all data from channels will be kept raw, no conversion applied. If many float are stored in file, you can gain from 3 to 4 times memory footprint To calculate value from channel, you can then use method .getChannelData()
filterChannelNames : bool, optional
flag to filter long channel names from its module names separated by ‘.’
compression : bool, optional
flag to activate data compression with blosc
metadata: int, optional, default = 2
Reading metadata has impact on performance, especially for mdf 4.x using xml. 2: minimal metadata reading (mostly channel blocks) 1: used for noDataLoading 0: all metadata reading
-
write3
(fileName=None)¶ Writes simple mdf 3.3 file
Parameters: fileName : str, optional
Name of file If file name is not input, written file name will be the one read with appended ‘_new’ string before extension
Notes
All channels will be converted to physical data, so size might be bigger than original file
-
-
mdfreader.mdf3reader.
polyConv
(data, conv)¶ apply polynomial conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
mdfreader.mdf3reader.
rationalConv
(data, conv)¶ apply rational conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
class
mdfreader.mdf3reader.
record
(dataGroup, channelGroup)¶ Bases:
list
- record class lists Channel classes,
- it is representing a channel group
Attributes
CGrecordLength (int) length of record from channel group block information in Byte recordLength (int) length of record from channels information in Byte numberOfRecords (int) number of records in data block recordID (int) recordID corresponding to channel group recordIDnumber (int) size of recordID dataGroup (int:) data group number channelGroup (int) channel group number numpyDataRecordFormat (list) list of numpy (dtype) for each channel dataRecordName (list) list of channel names used for recarray attribute definition master (dict) define name and number of master channel recordToChannelMatching (dict) helps to identify nested bits in byte channelNames (set) channel names to be stored, useful for low memory consumption but slow hiddenBytes (Bool, False by default) flag in case of non declared channels in record byte_aligned (Bool, True by default) flag for byte aligned record Methods
addChannel(info, channelNumber) loadInfo(info) readSortedRecord(fid, pointer, channelSet=None) readRecordBuf(buf, channelSet=None) readRecordBits(bita, channelSet=None) -
addChannel
(info, channelNumber)¶ add a channel in class
Parameters: info : mdfinfo3.info3 class
channelNumber : int
channel number in mdfinfo3.info3 class
-
loadInfo
(info)¶ gathers records related from info class
Parameters: info : mdfinfo3.info3 class
-
readRecordBits
(bita, channelSet=None)¶ read stream of record bits by bits in case of not aligned or hidden bytes
Parameters: buf : stream
stream of bytes read in file
channelSet : Set of str, optional
list of channel to read
Returns: rec : dict
returns dictionary of channel with its corresponding values
-
readRecordBuf
(buf, channelSet=None)¶ read stream of record bytes
Parameters: buf : stream
stream of bytes read in file
channelSet : Set of str, optional
list of channel to read
Returns: rec : dict
returns dictionary of channel with its corresponding values
-
readSortedRecord
(fid, pointer, channelSet=None)¶ reads record, only one channel group per datagroup
Parameters: fid : float
file identifier
pointer
position in file of data block beginning
channelSet : Set of str, optional
list of channel to read
Returns: rec : numpy recarray
contains a matrix of raw data in a recarray (attributes corresponding to channel name)
Notes
If channelSet is None, read data using numpy.core.records.fromfile that is rather quick. However, in case of large file, you can use channelSet to load only interesting channels or only one channel on demand, but be aware it might be much slower.
-
mdfreader.mdf3reader.
tabConv
(data, conv)¶ apply Tabular conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
mdfreader.mdf3reader.
tabInterpConv
(data, conv)¶ apply Tabular interpolation conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
mdfreader.mdf3reader.
textRangeTableConv
(data, conv)¶ apply text range table conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value
-
mdfreader.mdf3reader.
textTableConv
(data, conv)¶ apply text table conversion to data
Parameters: data : numpy 1D array
raw data to be converted to physical value
conv : mdfinfo3.info3 conversion block (‘CCBlock’) dict
Returns: converted data to physical value