trendify
trendify ¶
Provides top-level imports
ProductGenerator
module-attribute
¶
ProductGenerator = Callable[[Path], ProductList]
Callable method type. Users must provide a ProductGenerator
to map over raw data.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
Path
|
Workdir holding raw data (Should be one per run from a batch) |
required |
Returns:
Type | Description |
---|---|
ProductList
|
List of data products to be sorted and used to produce assets |
ProductList
module-attribute
¶
ProductList = List[SerializeAsAny[InstanceOf[DataProduct]]]
List of serializable DataProduct or child classes thereof
DataProduct
pydantic-model
¶
Bases: BaseModel
Base class for data products to be generated and handled.
Attributes:
Name | Type | Description |
---|---|---|
product_type |
str
|
Product type should be the same as the class name. The product type is used to search for products from a DataProductCollection. |
tags |
Tags
|
Tags to be used for sorting data. |
metadata |
dict[str, str]
|
A dictionary of metadata to be used as a tool tip for mousover in grafana |
Show JSON schema:
{
"additionalProperties": true,
"description": "Base class for data products to be generated and handled.\n\nAttributes:\n product_type (str): Product type should be the same as the class name.\n The product type is used to search for products from a [DataProductCollection][trendify.API.DataProductCollection].\n tags (Tags): Tags to be used for sorting data.\n metadata (dict[str, str]): A dictionary of metadata to be used as a tool tip for mousover in grafana",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
}
},
"required": [
"tags"
],
"title": "DataProduct",
"type": "object"
}
Config:
extra
:'allow'
Fields:
Validators:
-
_remove_computed_fields
product_type
pydantic-field
¶
product_type: str
Returns:
Type | Description |
---|---|
str
|
Product type should be the same as the class name. The product type is used to search for products from a DataProductCollection. |
__init_subclass__ ¶
__init_subclass__(**kwargs: Any) -> None
Registers child subclasses to be able to parse them from JSON file using the deserialize_child_classes method
Source code in src/trendify/API.py
append_to_list ¶
append_to_list(l: List)
Appends self to list.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
l
|
List
|
list to which |
required |
Returns:
Type | Description |
---|---|
Self
|
returns instance of |
deserialize_child_classes
classmethod
¶
deserialize_child_classes(key: str, **kwargs)
Loads json data to pydandic dataclass of whatever DataProduct child time is appropriate
Parameters:
Name | Type | Description | Default |
---|---|---|---|
key
|
str
|
json key |
required |
kwargs
|
dict
|
json entries stored under given key |
{}
|
Source code in src/trendify/API.py
DataProductCollection
pydantic-model
¶
DataProductCollection(**kwargs: Any)
Bases: BaseModel
A collection of data products.
Use this class to serialize data products to JSON, de-serialized them from JSON, filter the products, etc.
Attributes:
Name | Type | Description |
---|---|---|
elements |
ProductList
|
A list of data products. |
Show JSON schema:
{
"$defs": {
"DataProduct": {
"additionalProperties": true,
"description": "Base class for data products to be generated and handled.\n\nAttributes:\n product_type (str): Product type should be the same as the class name.\n The product type is used to search for products from a [DataProductCollection][trendify.API.DataProductCollection].\n tags (Tags): Tags to be used for sorting data.\n metadata (dict[str, str]): A dictionary of metadata to be used as a tool tip for mousover in grafana",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
}
},
"required": [
"tags"
],
"title": "DataProduct",
"type": "object"
}
},
"description": "A collection of data products.\n\nUse this class to serialize data products to JSON, de-serialized them from JSON, filter the products, etc.\n\nAttributes:\n elements (ProductList): A list of data products.",
"properties": {
"derived_from": {
"anyOf": [
{
"format": "path",
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Derived From"
},
"elements": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/DataProduct"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Elements"
}
},
"title": "DataProductCollection",
"type": "object"
}
Fields:
-
derived_from
(Path | None
) -
elements
(ProductList | None
)
Source code in src/trendify/API.py
add_products ¶
add_products(*products: DataProduct)
Parameters:
Name | Type | Description | Default |
---|---|---|---|
products
|
Tuple[DataProduct | ProductList, ...]
|
Products or lists of products to be appended to collection elements. |
()
|
collect_from_all_jsons
classmethod
¶
Loads all products from JSONs in the given list of directories.
If recursive is set to True
, the directories will be searched recursively
(this could lead to double counting if you pass in subdirectories of a parent).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
dirs
|
Tuple[Path, ...]
|
Directories from which to load data product JSON files. |
()
|
recursive
|
bool
|
whether or not to search each of the provided directories recursively for data product json files. |
False
|
Returns:
Type | Description |
---|---|
Type[Self] | None
|
Data product collection if JSON files are found. |
Source code in src/trendify/API.py
drop_products ¶
Removes products matching tag
and/or object_type
from collection elements.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tag
|
Tag | None
|
Tag for which data products should be dropped |
None
|
object_type
|
Type | None
|
Type of data product to drop |
None
|
Returns:
Type | Description |
---|---|
DataProductCollection
|
A new collection from which matching elements have been dropped. |
Source code in src/trendify/API.py
from_iterable
classmethod
¶
from_iterable(*products: Tuple[ProductList, ...])
Returns a new instance containing all of the products provided in the *products
argument.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
products
|
Tuple[ProductList, ...]
|
Lists of data products to combine into a collection |
()
|
Returns:
Type | Description |
---|---|
cls
|
A data product collection containing all of the provided products in the |
Source code in src/trendify/API.py
get_products ¶
Returns a new collection containing products matching tag
and/or object_type
.
Both tag
and object_type
default to None
which matches all products.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tag
|
Tag | None
|
Tag of data products to be kept. |
None
|
object_type
|
Type | None
|
Type of data product to keep. |
None
|
Returns:
Type | Description |
---|---|
DataProductCollection
|
A new collection containing matching elements. |
Source code in src/trendify/API.py
get_tags ¶
get_tags(data_product_type: Type[DataProduct] | None = None) -> set
Gets the tags related to a given type of DataProduct
. Parent classes will match all child class types.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data_product_type
|
Type[DataProduct] | None
|
type for which you want to get the list of tags |
None
|
Returns:
Type | Description |
---|---|
set
|
set of tags applying to the given |
Source code in src/trendify/API.py
make_grafana_panels
classmethod
¶
Processes collection of elements corresponding to a single tag. This method should be called on a directory containing jsons for which the products have been sorted.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
dir_in
|
Path
|
Directory from which to read data products (should be sorted first) |
required |
panel_dir
|
Path
|
Where to put the panel information |
required |
Source code in src/trendify/API.py
1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 |
|
process_single_tag_collection
classmethod
¶
process_single_tag_collection(
dir_in: Path, dir_out: Path, no_tables: bool, no_xy_plots: bool, no_histograms: bool, dpi: int
)
Processes collection of elements corresponding to a single tag. This method should be called on a directory containing jsons for which the products have been sorted.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
dir_in
|
Path
|
Input directory for loading assets |
required |
dir_out
|
Path
|
Output directory for assets |
required |
no_tables
|
bool
|
Suppresses table asset creation |
required |
no_xy_plots
|
bool
|
Suppresses xy plot asset creation |
required |
no_histograms
|
bool
|
Suppresses histogram asset creation |
required |
dpi
|
int
|
Sets resolution of asset output |
required |
Source code in src/trendify/API.py
1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 |
|
sort_by_tags
classmethod
¶
sort_by_tags(
dirs_in: List[Path], dir_out: Path, data_products_fname: str = DATA_PRODUCTS_FNAME_DEFAULT
)
Loads the data product JSON files from dirs_in
sorts the products.
Sorted products are written to smaller files in a nested directory structure under dir_out
.
A nested directory structure is generated according to the data tags.
Resulting product files are named according to the directory from which they were originally loaded.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
dirs_in
|
List[Path]
|
Directories from which the data product JSON files are to be loaded. |
required |
dir_out
|
Path
|
Directory to which the sorted data products will be written into a nested folder structure generated according to the data tags. |
required |
data_products_fname
|
str
|
Name of data products file |
DATA_PRODUCTS_FNAME_DEFAULT
|
Source code in src/trendify/API.py
sort_by_tags_single_directory
classmethod
¶
sort_by_tags_single_directory(
dir_in: Path, dir_out: Path, data_products_fname: str = DATA_PRODUCTS_FNAME_DEFAULT
)
Loads the data product JSON files from dir_in
and sorts the products.
Sorted products are written to smaller files in a nested directory structure under dir_out
.
A nested directory structure is generated according to the data tags.
Resulting product files are named according to the directory from which they were originally loaded.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
dir_in
|
List[Path]
|
Directories from which the data product JSON files are to be loaded. |
required |
dir_out
|
Path
|
Directory to which the sorted data products will be written into a nested folder structure generated according to the data tags. |
required |
data_products_fname
|
str
|
Name of data products file |
DATA_PRODUCTS_FNAME_DEFAULT
|
Source code in src/trendify/API.py
union
classmethod
¶
union(*collections: DataProductCollection)
Aggregates all of the products from multiple collections into a new larger collection.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
collections
|
Tuple[DataProductCollection, ...]
|
Data product collections for which the products should be combined into a new collection. |
()
|
Returns:
Type | Description |
---|---|
Type[Self]
|
A new data product collection containing all products from
the provided |
Source code in src/trendify/API.py
Format2D
pydantic-model
¶
Bases: HashableBase
Formatting data for matplotlib figure and axes
Attributes:
Name | Type | Description |
---|---|---|
title_fig |
Optional[str]
|
Sets figure title |
title_legend |
Optional[str]
|
Sets legend title |
title_ax |
Optional[str]
|
Sets axis title |
label_x |
Optional[str]
|
Sets x-axis label |
label_y |
Optional[str]
|
Sets y-axis label |
lim_x_min |
float | str | None
|
Sets x-axis lower bound |
lim_x_max |
float | str | None
|
Sets x-axis upper bound |
lim_y_min |
float | str | None
|
Sets y-axis lower bound |
lim_y_max |
float | str | None
|
Sets y-axis upper bound |
Show JSON schema:
{
"additionalProperties": false,
"description": "Formatting data for matplotlib figure and axes\n\nAttributes:\n title_fig (Optional[str]): Sets [figure title][matplotlib.figure.Figure.suptitle]\n title_legend (Optional[str]): Sets [legend title][matplotlib.legend.Legend.set_title]\n title_ax (Optional[str]): Sets [axis title][matplotlib.axes.Axes.set_title]\n label_x (Optional[str]): Sets [x-axis label][matplotlib.axes.Axes.set_xlabel]\n label_y (Optional[str]): Sets [y-axis label][matplotlib.axes.Axes.set_ylabel]\n lim_x_min (float | str | None): Sets [x-axis lower bound][matplotlib.axes.Axes.set_xlim]\n lim_x_max (float | str | None): Sets [x-axis upper bound][matplotlib.axes.Axes.set_xlim]\n lim_y_min (float | str | None): Sets [y-axis lower bound][matplotlib.axes.Axes.set_ylim]\n lim_y_max (float | str | None): Sets [y-axis upper bound][matplotlib.axes.Axes.set_ylim]",
"properties": {
"title_fig": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Fig"
},
"title_legend": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Legend"
},
"title_ax": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Ax"
},
"label_x": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label X"
},
"label_y": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label Y"
},
"lim_x_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Min"
},
"lim_x_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Max"
},
"lim_y_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Min"
},
"lim_y_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Max"
}
},
"title": "Format2D",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
-
title_fig
(Optional[str] | None
) -
title_legend
(Optional[str] | None
) -
title_ax
(Optional[str] | None
) -
label_x
(Optional[str] | None
) -
label_y
(Optional[str] | None
) -
lim_x_min
(float | str | None
) -
lim_x_max
(float | str | None
) -
lim_y_min
(float | str | None
) -
lim_y_max
(float | str | None
)
union_from_iterable
classmethod
¶
Gets the most inclusive format object (in terms of limits) from a list of Format2D
objects.
Requires that the label and title fields are identical for all format objects in the list.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
format2ds
|
Iterable[Format2D]
|
Iterable of |
required |
Returns:
Type | Description |
---|---|
Format2D
|
Single format object from list of objects. |
Source code in src/trendify/API.py
HistogramEntry
pydantic-model
¶
Bases: PlottableData2D
Use this class to specify a value to be collected into a matplotlib histogram.
Attributes:
Name | Type | Description |
---|---|---|
tags |
Tags
|
Tags used to sort data products |
value |
float | str
|
Value to be binned |
style |
HistogramStyle
|
Style of histogram display |
Show JSON schema:
{
"$defs": {
"Format2D": {
"additionalProperties": false,
"description": "Formatting data for matplotlib figure and axes\n\nAttributes:\n title_fig (Optional[str]): Sets [figure title][matplotlib.figure.Figure.suptitle]\n title_legend (Optional[str]): Sets [legend title][matplotlib.legend.Legend.set_title]\n title_ax (Optional[str]): Sets [axis title][matplotlib.axes.Axes.set_title]\n label_x (Optional[str]): Sets [x-axis label][matplotlib.axes.Axes.set_xlabel]\n label_y (Optional[str]): Sets [y-axis label][matplotlib.axes.Axes.set_ylabel]\n lim_x_min (float | str | None): Sets [x-axis lower bound][matplotlib.axes.Axes.set_xlim]\n lim_x_max (float | str | None): Sets [x-axis upper bound][matplotlib.axes.Axes.set_xlim]\n lim_y_min (float | str | None): Sets [y-axis lower bound][matplotlib.axes.Axes.set_ylim]\n lim_y_max (float | str | None): Sets [y-axis upper bound][matplotlib.axes.Axes.set_ylim]",
"properties": {
"title_fig": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Fig"
},
"title_legend": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Legend"
},
"title_ax": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Ax"
},
"label_x": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label X"
},
"label_y": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label Y"
},
"lim_x_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Min"
},
"lim_x_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Max"
},
"lim_y_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Min"
},
"lim_y_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Max"
}
},
"title": "Format2D",
"type": "object"
},
"HistogramStyle": {
"description": "Label and style data for generating histogram bars\n\nAttributes:\n color (str): Color of bars\n label (str|None): Legend entry\n histtype (str): Histogram type corresponding to matplotlib argument of same name\n alpha_edge (float): Opacity of bar edge\n alpha_face (float): Opacity of bar face\n linewidth (float): Line width of bar outline\n bins (int | list[int] | Tuple[int] | NDArray[Shape[\"*\"], int] | None): Number of bins (see [matplotlib docs][matplotlib.pyplot.hist])",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
},
"histtype": {
"default": "stepfilled",
"title": "Histtype",
"type": "string"
},
"alpha_edge": {
"default": 1,
"title": "Alpha Edge",
"type": "number"
},
"alpha_face": {
"default": 0.3,
"title": "Alpha Face",
"type": "number"
},
"linewidth": {
"default": 2,
"title": "Linewidth",
"type": "number"
},
"bins": {
"anyOf": [
{
"type": "integer"
},
{
"items": {
"type": "integer"
},
"type": "array"
},
{
"maxItems": 1,
"minItems": 1,
"prefixItems": [
{
"type": "integer"
}
],
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Bins"
}
},
"title": "HistogramStyle",
"type": "object"
}
},
"additionalProperties": false,
"description": "Use this class to specify a value to be collected into a matplotlib histogram.\n\nAttributes:\n tags (Tags): Tags used to sort data products\n value (float | str): Value to be binned\n style (HistogramStyle): Style of histogram display",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
},
"format2d": {
"anyOf": [
{
"$ref": "#/$defs/Format2D"
},
{
"type": "null"
}
],
"default": null
},
"value": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "Value"
},
"style": {
"$ref": "#/$defs/HistogramStyle",
"default": {
"color": "k",
"label": null,
"histtype": "stepfilled",
"alpha_edge": 1.0,
"alpha_face": 0.3,
"linewidth": 2.0,
"bins": null
}
}
},
"required": [
"tags",
"value"
],
"title": "HistogramEntry",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
-
value
(float | str
) -
tags
(Tags
) -
style
(HistogramStyle
)
HistogramStyle
pydantic-model
¶
Bases: HashableBase
Label and style data for generating histogram bars
Attributes:
Name | Type | Description |
---|---|---|
color |
str
|
Color of bars |
label |
str | None
|
Legend entry |
histtype |
str
|
Histogram type corresponding to matplotlib argument of same name |
alpha_edge |
float
|
Opacity of bar edge |
alpha_face |
float
|
Opacity of bar face |
linewidth |
float
|
Line width of bar outline |
bins |
int | list[int] | Tuple[int] | NDArray[Shape['*'], int] | None
|
Number of bins (see matplotlib docs) |
Show JSON schema:
{
"description": "Label and style data for generating histogram bars\n\nAttributes:\n color (str): Color of bars\n label (str|None): Legend entry\n histtype (str): Histogram type corresponding to matplotlib argument of same name\n alpha_edge (float): Opacity of bar edge\n alpha_face (float): Opacity of bar face\n linewidth (float): Line width of bar outline\n bins (int | list[int] | Tuple[int] | NDArray[Shape[\"*\"], int] | None): Number of bins (see [matplotlib docs][matplotlib.pyplot.hist])",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
},
"histtype": {
"default": "stepfilled",
"title": "Histtype",
"type": "string"
},
"alpha_edge": {
"default": 1,
"title": "Alpha Edge",
"type": "number"
},
"alpha_face": {
"default": 0.3,
"title": "Alpha Face",
"type": "number"
},
"linewidth": {
"default": 2,
"title": "Linewidth",
"type": "number"
},
"bins": {
"anyOf": [
{
"type": "integer"
},
{
"items": {
"type": "integer"
},
"type": "array"
},
{
"maxItems": 1,
"minItems": 1,
"prefixItems": [
{
"type": "integer"
}
],
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Bins"
}
},
"title": "HistogramStyle",
"type": "object"
}
Fields:
Marker
pydantic-model
¶
Bases: HashableBase
Defines marker for scattering to matplotlib
Attributes:
Name | Type | Description |
---|---|---|
color |
str
|
Color of line |
size |
float
|
Line width |
alpha |
float
|
Opacity from 0 to 1 (inclusive) |
zorder |
float
|
Prioritization |
label |
Union[str, None]
|
Legend label |
symbol |
str
|
Matplotlib symbol string |
Show JSON schema:
{
"additionalProperties": false,
"description": "Defines marker for scattering to matplotlib\n\nAttributes:\n color (str): Color of line\n size (float): Line width\n alpha (float): Opacity from 0 to 1 (inclusive)\n zorder (float): Prioritization \n label (Union[str, None]): Legend label\n symbol (str): Matplotlib symbol string",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"size": {
"default": 5,
"title": "Size",
"type": "number"
},
"alpha": {
"default": 1,
"title": "Alpha",
"type": "number"
},
"zorder": {
"default": 0,
"title": "Zorder",
"type": "number"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
},
"symbol": {
"default": ".",
"title": "Symbol",
"type": "string"
}
},
"title": "Marker",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
as_scatter_plot_kwargs ¶
Returns:
Type | Description |
---|---|
dict
|
dictionary of |
Source code in src/trendify/API.py
from_pen
classmethod
¶
Converts Pen to marker with the option to specify a symbol
Pen
pydantic-model
¶
Bases: HashableBase
Defines the pen drawing to matplotlib.
Attributes:
Name | Type | Description |
---|---|---|
color |
str
|
Color of line |
size |
float
|
Line width |
alpha |
float
|
Opacity from 0 to 1 (inclusive) |
zorder |
float
|
Prioritization |
label |
Union[str, None]
|
Legend label |
Show JSON schema:
{
"additionalProperties": false,
"description": "Defines the pen drawing to matplotlib.\n\nAttributes:\n color (str): Color of line\n size (float): Line width\n alpha (float): Opacity from 0 to 1 (inclusive)\n zorder (float): Prioritization \n label (Union[str, None]): Legend label",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"size": {
"default": 1,
"title": "Size",
"type": "number"
},
"alpha": {
"default": 1,
"title": "Alpha",
"type": "number"
},
"zorder": {
"default": 0,
"title": "Zorder",
"type": "number"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
}
},
"title": "Pen",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
as_scatter_plot_kwargs ¶
Returns kwargs dictionary for passing to matplotlib plot method
Source code in src/trendify/API.py
Point2D
pydantic-model
¶
Bases: XYData
Defines a point to be scattered onto xy plot.
Attributes:
Name | Type | Description |
---|---|---|
tags |
Tags
|
Tags to be used for sorting data. |
x |
float | str
|
X value for the point. |
y |
float | str
|
Y value for the point. |
marker |
Marker | None
|
Style and label information for scattering points to matplotlib axes. Only the label information is used in Grafana. Eventually style information will be used in grafana. |
metadata |
dict[str, str]
|
A dictionary of metadata to be used as a tool tip for mousover in grafana |
Show JSON schema:
{
"$defs": {
"Format2D": {
"additionalProperties": false,
"description": "Formatting data for matplotlib figure and axes\n\nAttributes:\n title_fig (Optional[str]): Sets [figure title][matplotlib.figure.Figure.suptitle]\n title_legend (Optional[str]): Sets [legend title][matplotlib.legend.Legend.set_title]\n title_ax (Optional[str]): Sets [axis title][matplotlib.axes.Axes.set_title]\n label_x (Optional[str]): Sets [x-axis label][matplotlib.axes.Axes.set_xlabel]\n label_y (Optional[str]): Sets [y-axis label][matplotlib.axes.Axes.set_ylabel]\n lim_x_min (float | str | None): Sets [x-axis lower bound][matplotlib.axes.Axes.set_xlim]\n lim_x_max (float | str | None): Sets [x-axis upper bound][matplotlib.axes.Axes.set_xlim]\n lim_y_min (float | str | None): Sets [y-axis lower bound][matplotlib.axes.Axes.set_ylim]\n lim_y_max (float | str | None): Sets [y-axis upper bound][matplotlib.axes.Axes.set_ylim]",
"properties": {
"title_fig": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Fig"
},
"title_legend": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Legend"
},
"title_ax": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Ax"
},
"label_x": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label X"
},
"label_y": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label Y"
},
"lim_x_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Min"
},
"lim_x_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Max"
},
"lim_y_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Min"
},
"lim_y_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Max"
}
},
"title": "Format2D",
"type": "object"
},
"Marker": {
"additionalProperties": false,
"description": "Defines marker for scattering to matplotlib\n\nAttributes:\n color (str): Color of line\n size (float): Line width\n alpha (float): Opacity from 0 to 1 (inclusive)\n zorder (float): Prioritization \n label (Union[str, None]): Legend label\n symbol (str): Matplotlib symbol string",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"size": {
"default": 5,
"title": "Size",
"type": "number"
},
"alpha": {
"default": 1,
"title": "Alpha",
"type": "number"
},
"zorder": {
"default": 0,
"title": "Zorder",
"type": "number"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
},
"symbol": {
"default": ".",
"title": "Symbol",
"type": "string"
}
},
"title": "Marker",
"type": "object"
}
},
"additionalProperties": false,
"description": "Defines a point to be scattered onto xy plot.\n\nAttributes:\n tags (Tags): Tags to be used for sorting data. \n x (float | str): X value for the point.\n y (float | str): Y value for the point.\n marker (Marker | None): Style and label information for scattering points to matplotlib axes.\n Only the label information is used in Grafana.\n Eventually style information will be used in grafana.\n metadata (dict[str, str]): A dictionary of metadata to be used as a tool tip for mousover in grafana",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
},
"format2d": {
"anyOf": [
{
"$ref": "#/$defs/Format2D"
},
{
"type": "null"
}
],
"default": null
},
"x": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "X"
},
"y": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "Y"
},
"marker": {
"anyOf": [
{
"$ref": "#/$defs/Marker"
},
{
"type": "null"
}
],
"default": {
"color": "k",
"size": 5.0,
"alpha": 1.0,
"zorder": 0.0,
"label": null,
"symbol": "."
}
}
},
"required": [
"tags",
"x",
"y"
],
"title": "Point2D",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
ProductType ¶
TableEntry
pydantic-model
¶
Bases: DataProduct
Defines an entry to be collected into a table.
Collected table entries will be printed in three forms when possible: melted, pivot (when possible), and stats (on pivot columns, when possible).
Attributes:
Name | Type | Description |
---|---|---|
tags |
Tags
|
Tags used to sort data products |
row |
float | str
|
Row Label |
col |
float | str
|
Column Label |
value |
float | str
|
Value |
unit |
str | None
|
Units for value |
metadata |
dict[str, str]
|
A dictionary of metadata to be used as a tool tip for mousover in grafana |
Show JSON schema:
{
"additionalProperties": false,
"description": "Defines an entry to be collected into a table.\n\nCollected table entries will be printed in three forms when possible: melted, pivot (when possible), and stats (on pivot columns, when possible).\n\nAttributes:\n tags (Tags): Tags used to sort data products\n row (float | str): Row Label\n col (float | str): Column Label\n value (float | str): Value\n unit (str | None): Units for value\n metadata (dict[str, str]): A dictionary of metadata to be used as a tool tip for mousover in grafana",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
},
"row": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "Row"
},
"col": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "Col"
},
"value": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "boolean"
}
],
"title": "Value"
},
"unit": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"title": "Unit"
}
},
"required": [
"tags",
"row",
"col",
"value",
"unit"
],
"title": "TableEntry",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
get_entry_dict ¶
load_and_pivot
classmethod
¶
load_and_pivot(path: Path)
Loads melted table from csv and pivots to wide form.
csv should have columns named 'row'
, 'col'
, and 'value'
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
Path
|
path to CSV file |
required |
Returns:
Type | Description |
---|---|
DataFrame | None
|
Pivoted data frame or elese |
Source code in src/trendify/API.py
pivot_table
classmethod
¶
pivot_table(melted: DataFrame)
Attempts to pivot melted row, col, value DataFrame into a wide form DataFrame
Parameters:
Name | Type | Description | Default |
---|---|---|---|
melted
|
DataFrame
|
Melted data frame having columns named |
required |
Returns:
Type | Description |
---|---|
DataFrame | None
|
pivoted DataFrame if pivot works else |
Source code in src/trendify/API.py
Trace2D
pydantic-model
¶
Bases: XYData
A collection of points comprising a trace. Use the Trace2D.from_xy constructor.
Attributes:
Name | Type | Description |
---|---|---|
points |
List[Point2D]
|
List of points. Usually the points would have null values
for |
pen |
Pen
|
Style and label information for drawing to matplotlib axes. Only the label information is used in Grafana. Eventually style information will be used in grafana. |
tags |
Tags
|
Tags to be used for sorting data. |
metadata |
dict[str, str]
|
A dictionary of metadata to be used as a tool tip for mousover in grafana |
Show JSON schema:
{
"$defs": {
"Format2D": {
"additionalProperties": false,
"description": "Formatting data for matplotlib figure and axes\n\nAttributes:\n title_fig (Optional[str]): Sets [figure title][matplotlib.figure.Figure.suptitle]\n title_legend (Optional[str]): Sets [legend title][matplotlib.legend.Legend.set_title]\n title_ax (Optional[str]): Sets [axis title][matplotlib.axes.Axes.set_title]\n label_x (Optional[str]): Sets [x-axis label][matplotlib.axes.Axes.set_xlabel]\n label_y (Optional[str]): Sets [y-axis label][matplotlib.axes.Axes.set_ylabel]\n lim_x_min (float | str | None): Sets [x-axis lower bound][matplotlib.axes.Axes.set_xlim]\n lim_x_max (float | str | None): Sets [x-axis upper bound][matplotlib.axes.Axes.set_xlim]\n lim_y_min (float | str | None): Sets [y-axis lower bound][matplotlib.axes.Axes.set_ylim]\n lim_y_max (float | str | None): Sets [y-axis upper bound][matplotlib.axes.Axes.set_ylim]",
"properties": {
"title_fig": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Fig"
},
"title_legend": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Legend"
},
"title_ax": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Ax"
},
"label_x": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label X"
},
"label_y": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label Y"
},
"lim_x_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Min"
},
"lim_x_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Max"
},
"lim_y_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Min"
},
"lim_y_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Max"
}
},
"title": "Format2D",
"type": "object"
},
"Marker": {
"additionalProperties": false,
"description": "Defines marker for scattering to matplotlib\n\nAttributes:\n color (str): Color of line\n size (float): Line width\n alpha (float): Opacity from 0 to 1 (inclusive)\n zorder (float): Prioritization \n label (Union[str, None]): Legend label\n symbol (str): Matplotlib symbol string",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"size": {
"default": 5,
"title": "Size",
"type": "number"
},
"alpha": {
"default": 1,
"title": "Alpha",
"type": "number"
},
"zorder": {
"default": 0,
"title": "Zorder",
"type": "number"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
},
"symbol": {
"default": ".",
"title": "Symbol",
"type": "string"
}
},
"title": "Marker",
"type": "object"
},
"Pen": {
"additionalProperties": false,
"description": "Defines the pen drawing to matplotlib.\n\nAttributes:\n color (str): Color of line\n size (float): Line width\n alpha (float): Opacity from 0 to 1 (inclusive)\n zorder (float): Prioritization \n label (Union[str, None]): Legend label",
"properties": {
"color": {
"default": "k",
"title": "Color",
"type": "string"
},
"size": {
"default": 1,
"title": "Size",
"type": "number"
},
"alpha": {
"default": 1,
"title": "Alpha",
"type": "number"
},
"zorder": {
"default": 0,
"title": "Zorder",
"type": "number"
},
"label": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label"
}
},
"title": "Pen",
"type": "object"
},
"Point2D": {
"additionalProperties": false,
"description": "Defines a point to be scattered onto xy plot.\n\nAttributes:\n tags (Tags): Tags to be used for sorting data. \n x (float | str): X value for the point.\n y (float | str): Y value for the point.\n marker (Marker | None): Style and label information for scattering points to matplotlib axes.\n Only the label information is used in Grafana.\n Eventually style information will be used in grafana.\n metadata (dict[str, str]): A dictionary of metadata to be used as a tool tip for mousover in grafana",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
},
"format2d": {
"anyOf": [
{
"$ref": "#/$defs/Format2D"
},
{
"type": "null"
}
],
"default": null
},
"x": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "X"
},
"y": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
}
],
"title": "Y"
},
"marker": {
"anyOf": [
{
"$ref": "#/$defs/Marker"
},
{
"type": "null"
}
],
"default": {
"color": "k",
"size": 5.0,
"alpha": 1.0,
"zorder": 0.0,
"label": null,
"symbol": "."
}
}
},
"required": [
"tags",
"x",
"y"
],
"title": "Point2D",
"type": "object"
}
},
"additionalProperties": false,
"description": "A collection of points comprising a trace.\nUse the [Trace2D.from_xy][trendify.API.Trace2D.from_xy] constructor.\n\nAttributes:\n points (List[Point2D]): List of points. Usually the points would have null values \n for `marker` and `format2d` fields to save space.\n pen (Pen): Style and label information for drawing to matplotlib axes.\n Only the label information is used in Grafana.\n Eventually style information will be used in grafana.\n tags (Tags): Tags to be used for sorting data.\n metadata (dict[str, str]): A dictionary of metadata to be used as a tool tip for mousover in grafana",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
},
"format2d": {
"anyOf": [
{
"$ref": "#/$defs/Format2D"
},
{
"type": "null"
}
],
"default": null
},
"points": {
"items": {
"$ref": "#/$defs/Point2D"
},
"title": "Points",
"type": "array"
},
"pen": {
"$ref": "#/$defs/Pen",
"default": {
"color": "k",
"size": 1.0,
"alpha": 1.0,
"zorder": 0.0,
"label": null
}
}
},
"required": [
"tags",
"points"
],
"title": "Trace2D",
"type": "object"
}
Config:
extra
:'forbid'
Fields:
x
pydantic-field
¶
y
pydantic-field
¶
from_xy
classmethod
¶
from_xy(
tags: Tags,
x: NDArray[Shape["*"], float],
y: NDArray[Shape["*"], float],
pen: Pen = Pen(),
format2d: Format2D = Format2D(),
)
Creates a list of Point2Ds from xy data and returns a new Trace2D product.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tags
|
Tags
|
Tags used to sort data products |
required |
x
|
NDArray[Shape['*'], float]
|
x values |
required |
y
|
NDArray[Shape['*'], float]
|
y values |
required |
pen
|
Pen
|
Style and label for trace |
Pen()
|
format2d
|
Format2D
|
Format to apply to plot |
Format2D()
|
Source code in src/trendify/API.py
plot_to_ax ¶
Plots xy data from trace to a matplotlib axes object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
ax
|
Axes
|
axes to which xy data should be plotted |
required |
propagate_format2d_and_pen ¶
propagate_format2d_and_pen(marker_symbol: str = '.') -> None
Propagates format and style info to all self.points
(in-place).
I thought this would be useful for grafana before I learned better methods for propagating the data.
It still may end up being useful if my plotting method changes. Keeping for potential future use case.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
marker_symbol
|
str
|
Valid matplotlib marker symbol |
'.'
|
Source code in src/trendify/API.py
XYData
pydantic-model
¶
Bases: PlottableData2D
Base class for children of DataProduct to be plotted ax xy data on a 2D plot
Show JSON schema:
{
"$defs": {
"Format2D": {
"additionalProperties": false,
"description": "Formatting data for matplotlib figure and axes\n\nAttributes:\n title_fig (Optional[str]): Sets [figure title][matplotlib.figure.Figure.suptitle]\n title_legend (Optional[str]): Sets [legend title][matplotlib.legend.Legend.set_title]\n title_ax (Optional[str]): Sets [axis title][matplotlib.axes.Axes.set_title]\n label_x (Optional[str]): Sets [x-axis label][matplotlib.axes.Axes.set_xlabel]\n label_y (Optional[str]): Sets [y-axis label][matplotlib.axes.Axes.set_ylabel]\n lim_x_min (float | str | None): Sets [x-axis lower bound][matplotlib.axes.Axes.set_xlim]\n lim_x_max (float | str | None): Sets [x-axis upper bound][matplotlib.axes.Axes.set_xlim]\n lim_y_min (float | str | None): Sets [y-axis lower bound][matplotlib.axes.Axes.set_ylim]\n lim_y_max (float | str | None): Sets [y-axis upper bound][matplotlib.axes.Axes.set_ylim]",
"properties": {
"title_fig": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Fig"
},
"title_legend": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Legend"
},
"title_ax": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Title Ax"
},
"label_x": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label X"
},
"label_y": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Label Y"
},
"lim_x_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Min"
},
"lim_x_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim X Max"
},
"lim_y_min": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Min"
},
"lim_y_max": {
"anyOf": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Lim Y Max"
}
},
"title": "Format2D",
"type": "object"
}
},
"additionalProperties": true,
"description": "Base class for children of DataProduct to be plotted ax xy data on a 2D plot",
"properties": {
"tags": {
"items": {
"anyOf": []
},
"title": "Tags",
"type": "array"
},
"metadata": {
"additionalProperties": {
"type": "string"
},
"default": {},
"title": "Metadata",
"type": "object"
},
"format2d": {
"anyOf": [
{
"$ref": "#/$defs/Format2D"
},
{
"type": "null"
}
],
"default": null
}
},
"required": [
"tags"
],
"title": "XYData",
"type": "object"
}
get_data_products ¶
get_data_products(
analysis: str = "workdir.products", tag: str = "trace_plots", product_type: str = "DataProduct"
)
Traces
parse-json | project "elements" | extend "label"="pen.label" | mv-expand "points" | extend "x"="points.x", "y"="points.y" | project "label", "x", "y" | pivot sum("y"), "x", "label" | project "label", "x", "y"
Source code in src/trendify/server.py
make_grafana_dashboard ¶
make_grafana_dashboard(
products_dir: Path, output_dir: Path, protocol: str, host: str, port: int, n_procs: int = 1
)
Makes a JSON file to import to Grafana for displaying tagged data tables, histograms and XY plots.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
products_dir
|
Path
|
Root directory into which products have been sorted by tag |
required |
output_dir
|
Path
|
Root directory into which Grafana dashboard and panal definitions will be written |
required |
n_procs
|
int
|
Number of parallel tasks used for processing data product tags |
1
|
protocol
|
str
|
Communication protocol for data server |
required |
host
|
str
|
Sever address for providing data to interactive dashboard |
required |
n_procs
|
int
|
Number of parallel processes |
1
|
Source code in src/trendify/API.py
make_include_files ¶
make_include_files(
root_dir: Path,
local_server_path: str | Path = None,
mkdocs_include_dir: str | Path = None,
heading_level: int | None = None,
)
Makes nested include files for inclusion into an MkDocs site.
Note
I recommend to create a Grafana panel and link to that from the MkDocs site instead.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
root_dir
|
Path
|
Directory for which the include files should be recursively generated |
required |
local_server_path
|
str | Path | None
|
What should the beginning of the path look like?
Use |
None
|
mkdocs_include_dir
|
str | Path | None
|
Path to be used for mkdocs includes.
This path should correspond to includ dir in |
None
|
Note:
Here is how to setup `mkdocs.yml` file to have an `include_dir` that can be used to
include generated markdown files (and the images/CSVs that they reference).
```
plugins:
- macros:
include_dir: run_for_record
```
Source code in src/trendify/API.py
1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 |
|
make_it_trendy ¶
make_it_trendy(
data_product_generator: ProductGenerator | None,
input_dirs: List[Path],
output_dir: Path,
n_procs: int = 1,
dpi_static_plots: int = 500,
no_static_tables: bool = False,
no_static_xy_plots: bool = False,
no_static_histograms: bool = False,
no_grafana_dashboard: bool = False,
no_include_files: bool = False,
protocol: str = "http",
server: str = "0.0.0.0",
port: int = 8000,
data_products_fname: str = DATA_PRODUCTS_FNAME_DEFAULT,
)
Maps data_product_generator
over dirs_in
to produce data product JSON files in those directories.
Sorts the generated data products into a nested file structure starting from dir_products
.
Nested folders are generated for tags that are Tuples. Sorted data files are named according to the
directory from which they were loaded.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data_product_generator
|
ProductGenerator | None
|
A callable function that returns a list of data products given a working directory. |
required |
input_dirs
|
List[Path]
|
Directories over which to map the |
required |
output_dir
|
Path
|
Directory to which the trendify products and assets will be written. |
required |
n_procs
|
int = 1
|
Number of processes to run in parallel. If |
1
|
dpi_static_plots
|
int = 500
|
Resolution of output plots when using matplotlib
(for |
500
|
no_static_tables
|
bool
|
Suppresses static assets from the |
False
|
no_static_xy_plots
|
bool
|
False
|
|
no_static_histograms
|
bool
|
Suppresses static assets from the |
False
|
no_grafana_dashboard
|
bool
|
Suppresses generation of Grafana dashboard JSON definition file |
False
|
no_include_files
|
bool
|
Suppresses generation of include files for importing static assets to markdown or LaTeX reports |
False
|
data_products_fname
|
str
|
File name to be used for storing generated data products |
DATA_PRODUCTS_FNAME_DEFAULT
|
Source code in src/trendify/API.py
1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 |
|
make_products ¶
make_products(
product_generator: Callable[[Path], DataProductCollection] | None,
data_dirs: List[Path],
n_procs: int = 1,
data_products_fname: str = DATA_PRODUCTS_FNAME_DEFAULT,
)
Maps product_generator
over dirs_in
to produce data product JSON files in those directories.
Sorts the generated data products into a nested file structure starting from dir_products
.
Nested folders are generated for tags that are Tuples. Sorted data files are named according to the
directory from which they were loaded.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
product_generator
|
ProductGenerator | None
|
A callable function that returns a list of data products given a working directory. |
required |
data_dirs
|
List[Path]
|
Directories over which to map the |
required |
n_procs
|
int = 1
|
Number of processes to run in parallel. If |
1
|
data_products_fname
|
str
|
File name to be used for storing generated data products |
DATA_PRODUCTS_FNAME_DEFAULT
|
Source code in src/trendify/API.py
make_tables_and_figures ¶
make_tables_and_figures(
products_dir: Path,
output_dir: Path,
dpi: int = 500,
n_procs: int = 1,
no_tables: bool = False,
no_xy_plots: bool = False,
no_histograms: bool = False,
)
Makes CSV tables and creates plots (using matplotlib).
Tags will be processed in parallel and output in nested directory structure under output_dir
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
products_dir
|
Path
|
Directory to which the sorted data products will be written |
required |
output_dir
|
Path
|
Directory to which tables and matplotlib histograms and plots will be written if
the appropriate boolean variables |
required |
n_procs
|
int = 1
|
Number of processes to run in parallel. If |
1
|
dpi
|
int = 500
|
Resolution of output plots when using matplotlib
(for |
500
|
no_tables
|
bool
|
Whether or not to collect the
|
False
|
no_xy_plots
|
bool
|
Whether or not to plot the |
False
|
no_histograms
|
bool
|
Whether or not to generate histograms of the
|
False
|
Source code in src/trendify/API.py
sort_products ¶
sort_products(
data_dirs: List[Path],
output_dir: Path,
n_procs: int = 1,
data_products_fname: str = DATA_PRODUCTS_FNAME_DEFAULT,
)
Loads the tagged data products from data_dirs
and sorts them (by tag) into a nested folder structure rooted at output_dir
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
data_dirs
|
List[Path]
|
Directories containing JSON data product files |
required |
output_dir
|
Path
|
Directory to which sorted products will be written |
required |
data_products_fname
|
str
|
File name in which the data products to be sorted are stored |
DATA_PRODUCTS_FNAME_DEFAULT
|