Auto Generated Documentation¶
Preprocess the data frame Author: Arjun Majumdar, Eren Cakmak Created: July, 2019
-
preprocess.
data_preprocessing
(data)¶ A function to perform data preprocessing and interpolation Expects ‘data’ as input which is the Pandas DataFrame to be processed
-
preprocess.
grouping_data
(processed_data)¶ A function to group all values for each ‘animal_id’ Input is ‘processed_data’ which is processed Pandas DataFrame Returns a dictionary where- key is animal_id, value in Pandas DataFrame for that ‘animal_id’
Preprocess the Pandas Data frame Author: Arjun Majumdar, Eren Cakmak Created: July, 2019
-
interpolation.
linear_interpolation
(data, threshold)¶ Function to interpolate missing values for ‘x’ and ‘y’ attributes in dataset. ‘threshold’ parameter decides the number of rows till which, data should NOT be deleted.
CSV parsing and writing.
This module provides classes that assist in the reading and writing of Comma Separated Value (CSV) files, and implements the interface described by PEP 305. Although many CSV files are simple to parse, the format is not formally defined by a stable specification and is subtle enough that parsing lines of a CSV file with something like line.split(“,”) is bound to fail. The module supports three basic APIs: reading, writing, and registration of dialects.
DIALECT REGISTRATION:
Readers and writers support a dialect argument, which is a convenient handle on a group of settings. When the dialect argument is a string, it identifies one of the dialects previously registered with the module. If it is a class or instance, the attributes of the argument are used as the settings for the reader or writer:
- class excel:
delimiter = ‘,’ quotechar = ‘”’ escapechar = None doublequote = True skipinitialspace = False lineterminator = ‘rn’ quoting = QUOTE_MINIMAL
SETTINGS:
- quotechar - specifies a one-character string to use as the
quoting character. It defaults to ‘”’.
- delimiter - specifies a one-character string to use as the
field separator. It defaults to ‘,’.
- skipinitialspace - specifies how to interpret whitespace which
immediately follows a delimiter. It defaults to False, which means that whitespace immediately following a delimiter is part of the following field.
- lineterminator - specifies the character sequence which should
terminate rows.
- quoting - controls when quotes should be generated by the writer.
It can take on any of the following module constants:
- csv.QUOTE_MINIMAL means only when required, for example, when a
field contains either the quotechar or the delimiter
csv.QUOTE_ALL means that quotes are always placed around fields. csv.QUOTE_NONNUMERIC means that quotes are always placed around
fields which do not parse as integers or floating point numbers.
csv.QUOTE_NONE means that quotes are never placed around fields.
- escapechar - specifies a one-character string used to escape
the delimiter when quoting is set to QUOTE_NONE.
- doublequote - controls the handling of quotes inside fields. When
True, two consecutive quotes are interpreted as one during read, and when writing, each quote character embedded in the data is written as two quotes
-
exception
csv.
Error
¶
-
class
csv.
Dialect
¶ Describe a CSV dialect.
This must be subclassed (see csv.excel). Valid attributes are: delimiter, quotechar, escapechar, doublequote, skipinitialspace, lineterminator, quoting.
-
class
csv.
excel
¶ Describe the usual properties of Excel-generated CSV files.
-
class
csv.
excel_tab
¶ Describe the usual properties of Excel-generated TAB-delimited files.
-
csv.
field_size_limit
()¶ - Sets an upper limit on parsed fields.
csv.field_size_limit([limit])
Returns old limit. If limit is not given, no new limit is set and the old limit is returned
-
csv.
reader
()¶ - csv_reader = reader(iterable [, dialect=’excel’]
[optional keyword args])
- for row in csv_reader:
process(row)
The “iterable” argument can be any object that returns a line of input for each iteration, such as a file object or a list. The optional “dialect” parameter is discussed below. The function also accepts optional keyword arguments which override settings provided by the dialect.
The returned object is an iterator. Each iteration returns a row of the CSV file (which can span multiple input lines).
-
csv.
writer
()¶ - csv_writer = csv.writer(fileobj [, dialect=’excel’]
[optional keyword args])
- for row in sequence:
csv_writer.writerow(row)
[or]
- csv_writer = csv.writer(fileobj [, dialect=’excel’]
[optional keyword args])
csv_writer.writerows(rows)
The “fileobj” argument can be any object that supports the file API.
-
csv.
register_dialect
()¶ Create a mapping from a string name to a dialect class. dialect = csv.register_dialect(name[, dialect[, **fmtparams]])
-
csv.
get_dialect
()¶ Return the dialect instance associated with name. dialect = csv.get_dialect(name)
-
csv.
list_dialects
()¶ Return a list of all know dialect names. names = csv.list_dialects()
-
class
csv.
Sniffer
¶ “Sniffs” the format of a CSV file (i.e. delimiter, quotechar) Returns a Dialect object.
-
sniff
(sample, delimiters=None)¶ Returns a dialect (or None) corresponding to the sample
-
-
csv.
unregister_dialect
()¶ Delete the name/dialect mapping associated with a string name. csv.unregister_dialect(name)
-
class
csv.
unix_dialect
¶ Describe the usual properties of Unix-generated CSV files.
Microsoft (MS) Excel I/O in Python. Load the MS Excel data into pandas dataframe. Author: Arjun Majumdar, Eren Cakmak Created: August, 2019
-
excel.
parse_excel
(path_to_file)¶ Function to read Excel file into a Pandas DataFrame- Expects complete path/relative path to CSV file along with file name Expects package ‘xlrd’ to be installed for this to work!
Extract features for individual moving entities Author: Arjun Majumdar, Eren Cakmak Created: July, 2019
-
absolute.
compute_absolute_features
(data_animal_id_groups)¶ Calculate absolute features for the data animal group-
-
absolute.
compute_average_acceleration
(data_animal_id_groups, fps)¶ A function to compute average acceleration of an animal based on fps (frames per second) parameter.
Formulas used are- Average Acceleration = (Final Speed - Initial Speed) / Total Time Taken
-
absolute.
compute_average_speed
(data_animal_id_groups, fps)¶ Average Speed-
A function to compute average speed of an animal based on fps (frames per second) parameter. Calculate the average speed of a mover, based on the pandas dataframe and a frames per second (fps) parameter
Formula used- Average Speed = Total Distance Travelled / Total Time taken
-
absolute.
compute_distance_and_direction
(data_animal_id_groups)¶ Calculate metric distance and direction-
Calculate the metric distance between two consecutive time frames/time stamps for each moving entity (in this case, fish)