pyrotoolbox
Version 1.5b
Version 1.5b
This is a collection of different tools which are useful for data processing of PyroScience data. This module contains functions to parse data and to re-calculate it.
Please install a suitable python-distribution first. We recommend Anaconda-Python.
Afterwards you can install pyrotoolbox in the “Anaconda Command Prompt” by typing
pip install pyrotoolbox
This module contains parser to load data from different logfile formats.
The “parse” function is able to detect all possible input formats.
The return is for all functions a dataframe containing the data and a dictionary containing the parsed metadata. Independent of the input format the columns and metadata-names should be identical. Other functions in this module expect these naming conventions.
Reads any pyroscience textfile. Not .pyr files! Returns a dataframe and a dict with metadata.
fname – path to the textfile
Loads and parses a Workbench file and returns a pandas DataFrame and a dictionary with metadata
fname – file name of the logfile
(DataFrame, metadata-dict)
Loads and parses a Workbench file of a fireplate and returns a pandas DataFrame and a dictionary with metadata
fname – path to the lofile
DataFrame, metadata-dict
Loads and parses a logfile from the PyroDeveloperTool
fname – path to the logfile
(DataFrame, metadata-dict)
parses all files matching the pattern (default * .txt) and returns 3 dictionaries
first dictionary is UID/Name-ChX -> List of Dataframes
second dictionary is UID/Name-ChX -> List of metadata-dicts
third dictionary is UID/Name-ChX -> List of filenames
pattern – files to load. Default: * .txt
Loads and parses a logfile from an AquapHOx-Logger
fname – path to the logfile
(DataFrame, metadata-dict)
Loads and parses a logfile from a FSGO2
fname – path to the logfile
(DataFrame, metadata-dict)
Loads and parses a logfile from the FDO2 Logger
fname – path to the logfile
(DataFrame, metadata-dict)
[1]:
from pyrotoolbox import parse
%matplotlib inline
[2]:
df, m = parse('ChannelData/A_Firesting Pro (4 Channels)_(A Ch.1)_pH.txt')
[3]:
df
[3]:
time_s | pH | dphi | signal_intensity | ambient_light | R | status | sample_temperature | |
---|---|---|---|---|---|---|---|---|
date_time | ||||||||
2024-12-11 08:31:03.546 | -0.876 | 7.495 | 27.502 | 938 | 0 | 1.0920 | 0 | 37.579 |
2024-12-11 08:31:03.647 | -0.774 | 7.495 | 27.500 | 939 | 0 | 1.0921 | 0 | 37.579 |
2024-12-11 08:31:03.992 | -0.429 | 7.496 | 27.509 | 939 | 0 | 1.0915 | 0 | 37.581 |
2024-12-11 08:31:04.147 | -0.274 | 7.496 | 27.510 | 938 | 0 | 1.0915 | 0 | 37.581 |
2024-12-11 08:31:04.246 | -0.175 | 7.496 | 27.509 | 938 | 0 | 1.0915 | 0 | 37.581 |
... | ... | ... | ... | ... | ... | ... | ... | ... |
2024-12-11 08:55:36.025 | 1471.604 | 7.504 | 27.509 | 936 | 0 | 1.0879 | 0 | 37.569 |
2024-12-11 08:55:36.126 | 1471.705 | 7.503 | 27.502 | 936 | 0 | 1.0884 | 0 | 37.569 |
2024-12-11 08:55:36.226 | 1471.805 | 7.503 | 27.503 | 936 | 0 | 1.0884 | 0 | 37.570 |
2024-12-11 08:55:36.326 | 1471.905 | 7.503 | 27.505 | 936 | 0 | 1.0882 | 0 | 37.570 |
2024-12-11 08:55:36.427 | 1472.006 | 7.503 | 27.502 | 936 | 0 | 1.0884 | 0 | 37.569 |
9982 rows × 8 columns
[4]:
df['pH'].plot()
[4]:
<Axes: xlabel='date_time'>
[5]:
m
[5]:
{'experiment_name': ' comment',
'experiment_description': '\n',
'software_version': 'Workbench V1.5.3.2466',
'device': 'FSP19 [A] FSPRO-4',
'device_serial': '21450119',
'uid': '2466C2055D64A687',
'firmware': '4.11:001',
'channel': 1,
'sensor_code': 'SHG7-597-623',
'settings': {'duration': '16 ms',
'intensity': '80%',
'amp': '400x',
'frequency': 3000,
'crc_enable': False,
'write_lock': False,
'auto_flash_duration': False,
'auto_amp': True,
'analyte': 'pH',
'fiber_type': '1 mm',
'temperature': 'external sensor',
'pressure': 'internal sensor',
'salinity': 7.5,
'fiber_length_mm': 1000},
'calibration': {'date_calibration_acid': datetime.datetime(2024, 12, 9, 0, 0),
'date_calibration_base': None,
'date_calibration_offset': None,
'R1': 1.420695,
'pH1': 2.266,
'temp1': 24.89,
'salinity1': 2.0,
'R2': 0.046,
'pH2': 14.0,
'temp2': 20.0,
'salinity2': 7.5,
'offset': 0.0,
'dphi_ref': 57.8,
'attenuation_coefficient': 0.0339,
'bkgdAmpl': 0.584,
'bkgdDphi': 0.0,
'dsf_dye': 0.9047,
'dtf_dye': -0.00567,
'pka': 8.319,
'slope': 1.087,
'bottom_t': -0.0159,
'top_t': -0.002465,
'slope_t': 0.0,
'pka_t': -0.01147,
'pka_is1': 2.54,
'pka_is2': 0.25}}
[6]:
m['settings']
[6]:
{'duration': '16 ms',
'intensity': '80%',
'amp': '400x',
'frequency': 3000,
'crc_enable': False,
'write_lock': False,
'auto_flash_duration': False,
'auto_amp': True,
'analyte': 'pH',
'fiber_type': '1 mm',
'temperature': 'external sensor',
'pressure': 'internal sensor',
'salinity': 7.5,
'fiber_length_mm': 1000}
[7]:
m['calibration']
[7]:
{'date_calibration_acid': datetime.datetime(2024, 12, 9, 0, 0),
'date_calibration_base': None,
'date_calibration_offset': None,
'R1': 1.420695,
'pH1': 2.266,
'temp1': 24.89,
'salinity1': 2.0,
'R2': 0.046,
'pH2': 14.0,
'temp2': 20.0,
'salinity2': 7.5,
'offset': 0.0,
'dphi_ref': 57.8,
'attenuation_coefficient': 0.0339,
'bkgdAmpl': 0.584,
'bkgdDphi': 0.0,
'dsf_dye': 0.9047,
'dtf_dye': -0.00567,
'pka': 8.319,
'slope': 1.087,
'bottom_t': -0.0159,
'top_t': -0.002465,
'slope_t': 0.0,
'pka_t': -0.01147,
'pka_is1': 2.54,
'pka_is2': 0.25}
[8]:
df, m = parse('ChannelData FirePlate/A_FirePlate-O2_(A Ch.1)_Oxygen.txt')
[9]:
df
[9]:
time_s | A02_oxygen_%O2 | A02_dphi | A02_signal_intensity | A02_ambient_light | A02_status | A03_oxygen_%O2 | A03_dphi | A03_signal_intensity | A03_ambient_light | ... | H11_signal_intensity | H11_ambient_light | H11_status | H12_oxygen_%O2 | H12_dphi | H12_signal_intensity | H12_ambient_light | H12_status | case_temperature | pressure | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
date_time | |||||||||||||||||||||
2024-09-04 09:26:12.371 | -1.460 | 20.662 | 20.581 | 101 | 37 | 0 | 20.539 | 20.691 | 99 | 34 | ... | 102 | 33 | 0 | 20.539 | 20.985 | 87 | -6 | 0 | 25.62 | 974 |
2024-09-04 09:26:15.380 | 1.549 | 20.391 | 20.718 | 101 | 37 | 0 | 20.628 | 20.643 | 99 | 34 | ... | 102 | 33 | 0 | 20.633 | 20.934 | 87 | -6 | 0 | 25.62 | 974 |
2024-09-04 09:26:18.396 | 4.565 | 20.608 | 20.610 | 101 | 37 | 0 | 20.597 | 20.663 | 99 | 34 | ... | 102 | 33 | 0 | 20.641 | 20.934 | 87 | -6 | 0 | 25.60 | 974 |
2024-09-04 09:26:21.404 | 7.573 | 20.554 | 20.636 | 101 | 37 | 0 | 20.654 | 20.632 | 99 | 34 | ... | 102 | 34 | 0 | 20.706 | 20.899 | 87 | -6 | 0 | 25.62 | 974 |
2024-09-04 09:26:24.419 | 10.588 | 20.745 | 20.541 | 101 | 37 | 0 | 20.637 | 20.643 | 99 | 34 | ... | 102 | 37 | 0 | 20.740 | 20.884 | 87 | -3 | 0 | 25.62 | 974 |
2024-09-04 09:26:27.438 | 13.607 | 20.725 | 20.549 | 101 | 37 | 0 | 20.743 | 20.587 | 99 | 34 | ... | 102 | 37 | 0 | 20.714 | 20.895 | 87 | -3 | 0 | 25.62 | 974 |
2024-09-04 09:26:30.424 | 16.593 | 20.479 | 20.675 | 101 | 37 | 0 | 20.488 | 20.717 | 99 | 34 | ... | 102 | 37 | 0 | 20.624 | 20.941 | 87 | -3 | 0 | 25.62 | 974 |
2024-09-04 09:26:33.441 | 19.610 | 20.576 | 20.625 | 101 | 37 | 0 | 20.558 | 20.681 | 99 | 34 | ... | 102 | 37 | 0 | 20.652 | 20.927 | 87 | -3 | 0 | 25.62 | 974 |
2024-09-04 09:26:36.448 | 22.617 | 20.877 | 20.472 | 101 | 37 | 0 | 20.541 | 20.690 | 99 | 34 | ... | 102 | 37 | 0 | 20.691 | 20.907 | 87 | -3 | 0 | 25.62 | 974 |
2024-09-04 09:26:39.456 | 25.626 | 20.542 | 20.647 | 101 | 37 | 0 | 20.527 | 20.702 | 99 | 34 | ... | 102 | 37 | 0 | 20.710 | 20.902 | 87 | -2 | 0 | 25.62 | 974 |
2024-09-04 09:26:42.464 | 28.633 | 20.571 | 20.630 | 101 | 37 | 0 | 20.702 | 20.610 | 99 | 34 | ... | 102 | 37 | 0 | 20.590 | 20.961 | 87 | -3 | 0 | 25.62 | 974 |
2024-09-04 09:26:45.477 | 31.646 | 20.666 | 20.579 | 101 | 37 | 0 | 20.611 | 20.654 | 99 | 34 | ... | 102 | 37 | 0 | 20.739 | 20.882 | 87 | -2 | 0 | 25.62 | 974 |
2024-09-04 09:26:48.484 | 34.653 | 20.623 | 20.601 | 101 | 37 | 0 | 20.709 | 20.604 | 99 | 34 | ... | 102 | 37 | 0 | 20.591 | 20.958 | 87 | -2 | 0 | 25.62 | 974 |
2024-09-04 09:26:51.493 | 37.662 | 20.581 | 20.622 | 101 | 37 | 0 | 20.575 | 20.672 | 99 | 34 | ... | 102 | 37 | 0 | 20.666 | 20.919 | 87 | -2 | 0 | 25.63 | 974 |
2024-09-04 09:26:54.513 | 40.682 | 20.740 | 20.543 | 101 | 37 | 0 | 20.668 | 20.627 | 99 | 34 | ... | 102 | 37 | 0 | 20.624 | 20.943 | 86 | -2 | 0 | 25.63 | 974 |
2024-09-04 09:26:57.507 | 43.677 | 20.535 | 20.648 | 101 | 37 | 0 | 20.753 | 20.584 | 99 | 34 | ... | 102 | 34 | 0 | 20.678 | 20.916 | 87 | -5 | 0 | 25.62 | 974 |
16 rows × 423 columns
[10]:
df.filter(regex='%O2').plot()
[10]:
<Axes: xlabel='date_time'>
[11]:
m['settings']
[11]:
{'duration': '1 ms',
'intensity': '40%',
'amp': '400x',
'frequency': 4000,
'crc_enable': False,
'write_lock': False,
'auto_flash_duration': True,
'auto_amp': True,
'analyte': 'oxygen',
'fiber_type': '1 mm',
'temperature': 'internal sensor',
'pressure': 'internal sensor',
'salinity': 7.5,
'fiber_length_mm': 0}
[12]:
import pandas as pd
pd.DataFrame(m['calibration'])
[12]:
A02 | A03 | A04 | A05 | A06 | A07 | A08 | A10 | A11 | A12 | ... | H02 | H03 | H04 | H05 | H06 | H07 | H09 | H10 | H11 | H12 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
date_calibration_high | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | ... | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 | 2024-09-04 00:00:00 |
date_calibration_zero | None | None | None | None | None | None | None | None | None | None | ... | None | None | None | None | None | None | None | None | None | None |
dphi100 | 20.615223 | 20.661667 | 20.648781 | 20.536888 | 20.556 | 20.665443 | 20.582445 | 20.501333 | 20.513332 | 20.47089 | ... | 20.465332 | 20.582666 | 20.542221 | 20.487556 | 20.521555 | 20.564335 | 20.399332 | 20.480223 | 20.585888 | 20.956335 |
dphi0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | ... | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 | 50.0 |
f | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | ... | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 | 0.842286 |
m | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | ... | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 | 0.087819 |
freq | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | ... | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 | 4000.0 |
tt | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | ... | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 | -0.000563 |
kt | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | ... | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 | 0.011441 |
bkgdAmpl | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | ... | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 | 0.04411 |
bkgdDphi | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
mt | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | ... | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 | -0.000329 |
pressure | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | ... | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 | 973.98999 |
temp100 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | ... | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 | 25.288 |
humidity | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | ... | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 | 40.0 |
temp0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | ... | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 | 20.0 |
percentO2 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | ... | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 | 20.95 |
17 rows × 84 columns
[ ]:
A collection of functions to re-calculate results from measurements.
The following functions can be used to calculate oxygen partial pressures from the phase angle dphi. All other oxygen units can be calculated from the partial pressure.
Module containing functions for oxygen unit conversions.
Calculate pO2 in hPa for a given phase-angle and temperature.
dphi – phase angle in °
temperature – in °C
dphi0 – phase angle at zero calibration point in °
dphi100 – phase angle at the upper calibration point in !
temp0 – temperature at zero calibration point in °C
temp100 – temperature at the upper calibration point in °C
pressure – pressure at the upper calibration point in mbar/hPa
humidity – relative humidity at the upper calibration point in %
percentO2 – oxygen content in the dry calibration gas (upper calibration point) in %
f – sensor constant
m – sensor constant
kt – sensor constant (in 1/K)
tt – sensor constant (in 1/K)
mt – sensor constant (in 1/K)
ft – sensor constant (in 1/K)
freq – modulation frequency in Hz
kwargs – to accept additional unused parameters. (allow passing **calibration without an error)
pO2 in hPa
Calculate pO2 in hPa for a given dphi, temperature and calibration.
phase angle in °
temperature – in °C
calibration – dictionary as generated by the parser functions (metadata[‘calibration’]).
dphi0 [°]
dphi100 [°]
temp0 [°C]
temp100 [°C]
pressure [mbar]
humidity [%]
percentO2 [%]
f
m
kt [1/K]
tt [1/K]
mt [1/K]
ft [1/K]
The following functions are used to convert between oxygen units. The partial pressure of oxygen (in hPa) is the central unit. All units can be converted toward pO2 and all units can be calculated from pO2. It is also the measured unit on all devices.
Module containing functions for oxygen unit conversions.
Convert any oxygen data to hPa.
Useful for people like me who prefer to look at oxygen data in hPa. This function takes the dataframe and metadata dict of the parser functions and returns the oxygen data in hPa. If you want to change the conditions of the conversions (e.g. assuming the experiment setup was wrong) use the convert_to_hPa function.
df – DataFrame as generated by the parser functions.
m – metadata-dict as generated by the parser functions.
Series with oxygen data in hPa.
A copy of the popular “i_only_think_in_hpa” function for fireplate data.
This was separated due to different format of fireplate data, but works very similar.
df – DataFrame as generated by the read_fireplate_workbench function.
m – metadata-dict as generated by the read_fireplate_workbench function.
DataFrame with oxygen data in hPa.
Convert any oxygen unit to hPa.
data – any oxygen data produced by a pyroscience device.
unit – unit of the data. Has to be one of (‘oxygen_hPa’, ‘oxygen_torr’, ‘oxygen_%O2’, ‘oxygen_%airsat’, ‘oxygen_µM’, ‘oxygen_µg/L’, ‘oxygen_mg/L’, ‘oxygen_mL/L’).
temperature – sample temperature. Used for all units except ‘oxygen_hPa’, ‘oxygen_torr’ and ‘oxygen_%O2’.
pressure – gas-pressure of or above the sample. Used for all units except ‘oxygen_hPa’ and ‘oxygen_torr’
salinity – salinity of the sample in g/L. Used for all dissolved oxygen units.
Convert hPa to torr.
data – hPa data to convert.
Convert hPa to %O2.
data – hPa data to convert.
pressure – gas pressure of the sample in hPa.
Convert hPa to %airsat. This is a unit for water samples and assumes 100% relative humidity.
data – hPa data to convert.
pressure – gas pressure above the sample in hPa.
temperature – temperature of the sample in °C.
Convert hPa to µM. This is a unit for water sample.
data – hPa data to convert.
temperature – temperature of the sample in °C.
salinity – salinity of the sample in g/L.
Convert hPa to mg/L. This is a unit for water sample.
data – hPa data to convert.
temperature – temperature of the sample in °C.
salinity – salinity of the sample in g/L.
The following functions are implemented do to the above documented calculations. They might still be useful for your own unit conversions
Module containing functions for oxygen unit conversions.
Calculates the vapour pressure of water at given Temperature (°C), return in hPa
#This is Equation (6) from Murray, F.W. 1967. On the computation of saturation vapour pressure. J. Applied Meteorology 6: 203-204
temperature – Temperature in °C
vapour pressure in hPa
Calculate the oxygen solubility in µM for water with given temperature and salinity.
The formula and constants from “From Garcia 1992: Oxygen solubility in seawater: better fitting equations” are used.
temperature – water temperature (°C)
salinity – salinity of water (g/L)
oxygen solubility in µM
calculate the partial pressure of oxygen in air with a given water saturation and total pressure
pressure – ambient pressure in hPa
T – Temperature in °C
water_sat – water saturation of the air in % (0-100)
percentO2 – percentage of oxygen in air in % (0-100). Default 20.95
partial pressure of oxygen in hPa
[ ]:
from pyrotoolbox import parse
%matplotlib inline
from pyrotoolbox.oxygen import hPa_to_mgL
[3]:
# Load AquapHOx-L data
df, m = parse('AphoxData/21270013.txt')
[8]:
# Convert hPa to mg/L for a salinity of 32 g/L
[9]:
hPa_to_mgL(df['oxygen_hPa'], df['sample_temperature'], 32)
[9]:
date_time
2024-07-04 11:21:14 7.796528
2024-07-04 11:21:44 7.804866
2024-07-04 11:22:14 7.803978
2024-07-04 11:22:44 7.860132
2024-07-04 11:23:14 7.687174
...
2024-07-04 19:33:44 7.388505
2024-07-04 19:34:14 7.383316
2024-07-04 19:34:44 7.377246
2024-07-04 19:35:14 7.366197
2024-07-04 19:35:44 7.359155
Length: 990, dtype: float64
[1]:
from pyrotoolbox.parsers import parse
from pyrotoolbox.oxygen import *
[2]:
df, m = parse('datademo4/A_Firesting Pro (4 Channels)_(A Ch.1)_Oxygen.txt')
[3]:
df = df.iloc[80:].copy()
[4]:
df['oxygen_hPa'] = i_only_think_in_hpa(df, m)
[5]:
df['oxygen_hPa_2'] = convert_to_hPa(df['oxygen_%O2'], unit='oxygen_%O2',
temperature=m['settings']['temperature'],
pressure=df['pressure'])
[6]:
df['oxygen_torr'] = hPa_to_torr(df['oxygen_hPa'])
[7]:
df['oxygen_%O2'] = hPa_to_percentO2(df['oxygen_hPa'], df['pressure'])
[8]:
df['oxygen_%airsat'] = hPa_to_percent_airsat(df['oxygen_hPa'], df['pressure'], temperature=20)
[9]:
df['oxygen_mgL'] = hPa_to_mgL(df['oxygen_hPa'], temperature=20, salinity=7.5)
[10]:
df['oxygen_uM'] = hPa_to_uM(df['oxygen_hPa'], temperature=20, salinity=7.5)
Example: user has logged data with a fixed-T setting of 37°C and salinity of 7.5g/L, but the real temperature was 32°C and real salinity was 11g/L.
[11]:
po2 = i_only_think_in_hpa(df, m)
hPa_to_mgL(po2, temperature=32, salinity=11)
[11]:
date_time
2023-04-03 13:16:28.485 0.247168
2023-04-03 13:16:29.486 1.704633
2023-04-03 13:16:30.486 2.777244
2023-04-03 13:16:31.486 3.508465
2023-04-03 13:16:32.486 3.947728
2023-04-03 13:16:33.485 4.229732
2023-04-03 13:16:34.485 4.379028
2023-04-03 13:16:35.485 4.378365
2023-04-03 13:16:36.485 4.379360
2023-04-03 13:16:37.485 4.419836
2023-04-03 13:16:38.485 4.491167
2023-04-03 13:16:39.486 4.563492
2023-04-03 13:16:40.487 4.635486
Name: oxygen_hPa, dtype: float64
[ ]:
Module to calculate pH for devices from Pyroscience.
The following functions are only valid for devices with Firmware >= 4.10 (released 2023).
Calculate pH from R, temperature, salinity and sensor constants. See also function calculate_pH_from_calibration.
R – R-value from the sensor
temperature – temperature of the sample in °C
salinity – salinity of the sample in g/L
top – material constant
bottom – material constant
pka – material constant
slope – material constant
pka_t – material constant
bottom_t – material constant
top_t – material constant
slope_t – material constant
pka_is1 – material constant
pka_is2 – material constant
offset – offset in pH-units (from 3rd calibration point)
kwargs – kwargs are ignored
apply a pH calibration to measurement data for FW >= 410
The calibration parameters can be passed from the calibration metadata.
Example usage: apply_pH_calibration(data[‘R’], data[‘temp’], data[‘salinity’], m[‘calibration’])
R – R-value from the sensor
temperature – temperature of the sample in °C
salinity – salinity of the sample in g/L
calibration – calibration dictionary as created by the parsers
kwargs – kwargs are inserted into the calibration and override the values
calculated pH values
Apply an interpolated pH calibration to measurement data
The top and bottom value is calculated for every passed calibration and fitted linear over time. For every measurement point an individual value of top and bottom is calculated.
R – data for “R”
temperature – overrides temp values from df. Single value or iterable with same length as df is required
salinity – Single value or iterable with same length as df is required
calibrations – dict in format {timestring: calib_data} e.g. {‘2019-05-05 13:10:00’: {pH1: 4, temp1: 22.08, …}}, or a list of calibration-dicts
R – overrides R values from df. Single value or iterable with same length as df is required
return_fits – return the fits of top, bottom and offset instead
calculated pH-values
apply prospective pH drift compensation to a pH measurement.
The effect on the sensors is assumed to be only a decrease in Top.
R – R-values, pandas Series or DataFrame with time index
temperature – temperature data
salinity – salinity data
calibration – calibration dictionary as created by the parser functions
start_timestamp – optional, time of the calibration. By default, the date_calibration_acid value is used
d0 – first coefficient of the temperature dependent drift
d1 – second coefficient of the temperature dependent drift
d2 – third coefficient of the temperature dependent drift
d3 – fourth coefficient of the temperature dependent drift
start_timestamp – optional, start of the drift. If omitted the date of the last acid calibration is used
array of pH values
[1]:
import matplotlib.pyplot as plt
%matplotlib inline
from pyrotoolbox.parsers import parse
from pyrotoolbox.pH import (calculate_pH, calculate_pH_from_calibration, calculate_pH_from_interpolated_calibration,
calculate_pH_with_prospective_drift_compensation)
[2]:
pH_hl_1, m_pH_hl_1 = parse('rovdata/PH2ROV1.txt')
pH_hl_2, m_pH_hl_2 = parse('rovdata/PH2ROV2.txt')
[3]:
pH_hl_1
[3]:
status | dphi | sample_temperature | case_temperature | signal_intensity | ambient_light | pressure | humidity | pH | R | time_s | |
---|---|---|---|---|---|---|---|---|---|---|---|
date_time | |||||||||||
2024-04-08 13:55:32 | 0.0 | 21.674 | 26.864 | 27.20 | 293.129 | 27.902 | 983.661 | 34.791 | 4.820 | 1.576412 | 0.0 |
2024-04-08 13:56:32 | 0.0 | 21.680 | 26.916 | 26.97 | 294.613 | 26.226 | 984.164 | 35.085 | NaN | 1.577852 | 60.0 |
2024-04-08 13:57:32 | 0.0 | 21.686 | 27.272 | 26.80 | 291.169 | 26.382 | 984.078 | 35.250 | NaN | 1.578747 | 120.0 |
2024-04-08 13:58:32 | 0.0 | 21.689 | 27.397 | 26.71 | 290.793 | 26.150 | 983.909 | 35.310 | NaN | 1.579240 | 180.0 |
2024-04-08 13:59:32 | 0.0 | 21.695 | 27.459 | 26.64 | 290.417 | 26.679 | 983.721 | 35.383 | NaN | 1.579230 | 240.0 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
2024-07-09 17:25:32 | 0.0 | 33.999 | 29.246 | 26.80 | 162.006 | 32.015 | 1042.066 | 43.950 | 8.016 | 0.714338 | 7961400.0 |
2024-07-09 17:26:32 | 0.0 | 33.610 | 28.997 | 27.93 | 163.530 | 31.317 | 1042.987 | 42.979 | 8.004 | 0.727966 | 7961460.0 |
2024-07-09 17:27:32 | 0.0 | 33.761 | 28.950 | 28.68 | 162.294 | 29.670 | 1044.064 | 42.391 | 8.017 | 0.717720 | 7961520.0 |
2024-07-09 17:28:32 | 0.0 | 33.905 | 28.877 | 29.12 | 160.626 | 44.748 | 1045.449 | 42.130 | 8.028 | 0.709191 | 7961580.0 |
2024-07-09 17:29:32 | 0.0 | 34.128 | 29.732 | 29.44 | 158.054 | 47.130 | 1046.408 | 41.959 | 8.027 | 0.697615 | 7961640.0 |
132695 rows × 11 columns
[4]:
calculate_pH_from_calibration?
Signature:
calculate_pH_from_calibration(
R,
temperature,
salinity,
calibration: dict,
**kwargs,
)
Docstring:
apply a pH calibration to measurement data for FW >= 410
The calibration parameters can be passed from the calibration metadata.
Example usage:
apply_pH_calibration(data['R'], data['temp'], data['salinity'], m['calibration'])
:param R: R-value from the sensor
:param temperature: temperature of the sample in °C
:param salinity: salinity of the sample in g/L
:param calibration: calibration dictionary as created by the parsers
:param kwargs: kwargs are inserted into the calibration and override the values
:return: calculated pH values
File: ~/Programmieren/pyrotoolbox/pyrotoolbox/pH.py
Type: function
[5]:
pH_hl_1['pH_endcal'] = calculate_pH_from_calibration(pH_hl_1['R'],
pH_hl_1['sample_temperature'],
m_pH_hl_1['settings']['salinity'],
m_pH_hl_2['calibration'])
/home/christoph/.venvs/pyro/lib/python3.12/site-packages/pandas/core/arraylike.py:399: RuntimeWarning: invalid value encountered in log10
result = getattr(ufunc, method)(*inputs, **kwargs)
[6]:
pH_hl_1
[6]:
status | dphi | sample_temperature | case_temperature | signal_intensity | ambient_light | pressure | humidity | pH | R | time_s | pH_endcal | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
date_time | ||||||||||||
2024-04-08 13:55:32 | 0.0 | 21.674 | 26.864 | 27.20 | 293.129 | 27.902 | 983.661 | 34.791 | 4.820 | 1.576412 | 0.0 | NaN |
2024-04-08 13:56:32 | 0.0 | 21.680 | 26.916 | 26.97 | 294.613 | 26.226 | 984.164 | 35.085 | NaN | 1.577852 | 60.0 | NaN |
2024-04-08 13:57:32 | 0.0 | 21.686 | 27.272 | 26.80 | 291.169 | 26.382 | 984.078 | 35.250 | NaN | 1.578747 | 120.0 | NaN |
2024-04-08 13:58:32 | 0.0 | 21.689 | 27.397 | 26.71 | 290.793 | 26.150 | 983.909 | 35.310 | NaN | 1.579240 | 180.0 | NaN |
2024-04-08 13:59:32 | 0.0 | 21.695 | 27.459 | 26.64 | 290.417 | 26.679 | 983.721 | 35.383 | NaN | 1.579230 | 240.0 | NaN |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
2024-07-09 17:25:32 | 0.0 | 33.999 | 29.246 | 26.80 | 162.006 | 32.015 | 1042.066 | 43.950 | 8.016 | 0.714338 | 7961400.0 | 7.864503 |
2024-07-09 17:26:32 | 0.0 | 33.610 | 28.997 | 27.93 | 163.530 | 31.317 | 1042.987 | 42.979 | 8.004 | 0.727966 | 7961460.0 | 7.849334 |
2024-07-09 17:27:32 | 0.0 | 33.761 | 28.950 | 28.68 | 162.294 | 29.670 | 1044.064 | 42.391 | 8.017 | 0.717720 | 7961520.0 | 7.864751 |
2024-07-09 17:28:32 | 0.0 | 33.905 | 28.877 | 29.12 | 160.626 | 44.748 | 1045.449 | 42.130 | 8.028 | 0.709191 | 7961580.0 | 7.878145 |
2024-07-09 17:29:32 | 0.0 | 34.128 | 29.732 | 29.44 | 158.054 | 47.130 | 1046.408 | 41.959 | 8.027 | 0.697615 | 7961640.0 | 7.879995 |
132695 rows × 12 columns
[7]:
fig, ax = plt.subplots()
ax.plot(pH_hl_1['pH']['2024-04-10 15:00':], label='startcal')
ax.plot(pH_hl_1['pH_endcal']['2024-04-10 15:00':], label='endcal')
ax.legend()
ax.set_ylabel('pH')
fig.tight_layout()
[8]:
pH_hl_1['pH_fixedT25'] = calculate_pH_from_calibration(pH_hl_1['R'],
25,
m_pH_hl_1['settings']['salinity'],
m_pH_hl_1['calibration'])
[9]:
pH_hl_1['pH_devT'] = calculate_pH_from_calibration(pH_hl_1['R'],
pH_hl_1['case_temperature'],
m_pH_hl_1['settings']['salinity'],
m_pH_hl_1['calibration'])
/home/christoph/.venvs/pyro/lib/python3.12/site-packages/pandas/core/arraylike.py:399: RuntimeWarning: invalid value encountered in log10
result = getattr(ufunc, method)(*inputs, **kwargs)
[10]:
pH_hl_1['pH_PSU40'] = calculate_pH_from_calibration(pH_hl_1['R'],
pH_hl_1['sample_temperature'],
40,
m_pH_hl_1['calibration'])
/home/christoph/.venvs/pyro/lib/python3.12/site-packages/pandas/core/arraylike.py:399: RuntimeWarning: invalid value encountered in log10
result = getattr(ufunc, method)(*inputs, **kwargs)
[11]:
fig, ax = plt.subplots()
ax.plot(pH_hl_1['pH']['2024-04-10 15:00':], label='startcal')
ax.plot(pH_hl_1['pH_fixedT25']['2024-04-10 15:00':], label='fixed T=25')
ax.plot(pH_hl_1['pH_devT']['2024-04-10 15:00':], label='device T')
ax.plot(pH_hl_1['pH_PSU40']['2024-04-10 15:00':], label='40PSU')
ax.legend()
ax.set_ylabel('pH')
fig.tight_layout()
[12]:
pH_hl_1['pH_interpol_corr'] = calculate_pH_from_interpolated_calibration(
pH_hl_1['R'],
pH_hl_1['sample_temperature'],
35,
{'2024-04-08 13:35': m_pH_hl_1['calibration'],
'2024-07-09 13:00': m_pH_hl_2['calibration']})
/home/christoph/.venvs/pyro/lib/python3.12/site-packages/pandas/core/arraylike.py:399: RuntimeWarning: invalid value encountered in log10
result = getattr(ufunc, method)(*inputs, **kwargs)
[13]:
pH_hl_1['pH_corr_prosp'] = calculate_pH_with_prospective_drift_compensation(
pH_hl_1['R'],
pH_hl_1['sample_temperature'],
35,
m_pH_hl_1['calibration'], 4.3652*10**10, 9.0761*10**3,0,0)
/home/christoph/.venvs/pyro/lib/python3.12/site-packages/pandas/core/arraylike.py:399: RuntimeWarning: invalid value encountered in log10
result = getattr(ufunc, method)(*inputs, **kwargs)
[14]:
fig, ax = plt.subplots()
ax.plot(pH_hl_1['pH']['2024-04-10 15:00':], label='startcal')
ax.plot(pH_hl_1['pH_endcal']['2024-04-10 15:00':], label='endcal')
ax.plot(pH_hl_1['pH_interpol_corr']['2024-04-10 15:00':], label='Interpolated')
ax.plot(pH_hl_1['pH_corr_prosp']['2024-04-10 15:00':], label='Prospective')
ax.legend()
ax.set_ylabel('pH')
fig.tight_layout()
[ ]:
FireResponse is a graphical tool to extract response times from measurements. It can be started by running “FireResponse” in a terminal.
PyroHtmlReporter is still in active development. Anything might change!
A tool to generate html reports of measurement data. The report contains a summary of the measurement, the settings and calibration registers and an interactive plot of the most important data. The reports have the filename of the input file with an additional “_report.html”. If more than one file is passed at once the data is combined into a “summary.html” file.
create short shareable reports of measurements
to have a quick look on measurement data
quick debugging
The reports are standalone html files and can be viewed in any browser.
PyroHtmlReporter my_measurement_data.txt
This results in a file “my_measurement_data_report.html”.
PyroHtmlReporter my_measurement_data1.txt my_measurement_data2.txt
This results in the files “my_measurement_data1_report.html”, “my_measurement_data2_report.html” and “summary.html”.