mypythontools

Python versions PyPI version Language grade: Python Documentation Status License: MIT Codecov

Some tools/functions/snippets/files used across projects.

It’s called mypythontools, but it’s also made for you…

Can be used as python library. Things like building the application with pyinstaller, incrementing version, generating rst files for sphinx docs, pushing to github or deploying to PyPi or other CI/CD functionality, creating config module or plot data is matter of calling one function or clicking one button (e.g. Vs code task).

Many projects - one codebase.

If you are not sure whether structure of your app will work with this code, check project-starter on github in content folder.

Paths are inferred, but if you have atypical structure or have more projects in cwd, use mypythontools.paths.set_paths() with path arguments.

There is also some extra stuff, that is not bundled via PyPI (cookiecutter, CSS for readthedocs etc.), such a content is under the Content topic.

Installation

Python >=3.6 (Python 2 is not supported).

Install just with:

pip install mypythontools

Content

There are a lot of stuff that’s not in python library (installable via pip), but still on github repository.

Link where you can find find that content:

https://github.com/Malachov/mypythontools/tree/master/content

Link where you can read about how to use it:

https://mypythontools.readthedocs.io/#content

Some examples of what you can find only on github

project-starter

Project scaffolding fast and easy.

Download project-starter from

https://github.com/Malachov/mypythontools/tree/master/content/project-starter

(You may need to download all mypythontools repository in zip)

And start developing.

In your IDE do bulk renaming across files and replace SET_YOUR_APP_NAME with name of your app / library.

This starter is for vue-eel applications (desktop as well as web) but also for python libraries that will be stored on PyPi.

If it’s python library, delete gui folder.

If it’s app with gui, delete setup.py, __init__.py and remove Installation and Modules from README. Also from assets delete mdi.css if not using icons and formulate.css if not using vue formulate. In package.json uncomment library you will be using.

Install used python libraries via pip install -r requirements.txt and install JS libraries as well with npm install in folder where package.json is.

To run an app in develop mode, you have to run both frontent as well as python. Run frontent with debugging app.py (do not run, just debug). Then run frontend with npm run serve in gui folder (or use Task explorer if using VS Code). Open your favourite browser and open

http://localhost:8080

It’s recommended to use Vue.js devtools extension where you can see what component is on cursor, edit props values or see list of all used mutations.

In opened app, there is a little help button where there is simple overview about how to develop with these tools.

Delete is faster than write, so there is many working examples like for example plot, various formating (flex row, flex column), settings panel, function call from python to js and vice versa or automatic alerting from python. If you want to see how some example is working, just use ctrl + F in IDE or check components for it’s props.

This is how the example looks like

project-starter-gui

Docs

It includes docs structure for sphinx docs generations from docstrings. Edit first few lines in conf.py, index.rst, navi.html and if you want, add static files like logo to _static.

Usually used with https://readthedocs.org/ free hosting that trigger deploy automatically after pushing to master. Because of correct relative redirecting in index.rst and navi.html use for readthedocs /path/ before relative link to ohter module.

It’s also necassary to generate other modules rst files for other pages but it’s if using push_script.py as CI/CD (see below), it’s generated automatically via apidoc.

It’s recommended to use with sphinx-alabaster-css (see below).

CI/CD

It also include github actions yml file for pushing codecov coverage file (without token) and .travis.yml for Travis CI. You don’t need those files and can you can delete it.

There is file push_script.py in folder utils which personally is better (faster) than travis and can do most of what you want from CI like run pipeline - running tests / generate docs / increment version / push to github / push to pypi.

Check utils module for more informations.

IDE files

It also include some default project specific

settings for VS Code. You can also delete it.

If developing py - vue - eel app this is the recommended way, but if you want to build just what is necessary from scratch, you can use this tutorial

https://mypythontools.readthedocs.io/mypythontools.pyvueeel.html#mypythontools.pyvueeel.help_starter_pack_vue_app

requirements

Install many libraries at once (no need for Anaconda). Download requirements.txt file from https://github.com/Malachov/mypythontools/tree/master/content/requirements and in that folder use:

pip install -r requirements.txt

It’s good for python libraries that other users with different versions of libraries will use. If not standalone application where freezing into virtual env is good idea - here is possible to use this requirements with using –upgrade from time to time to be sure that your library will be working for up to date version of dependencies.

sphinx-alabaster-css

It’s good idea to generate documentation from code. If you are useing sphinx and alabaster theme, you can use this css file for formating.

Tested on readthedocs hosting (recommended).

CSS are served from github and it’s possible to change on one place and edit how all projects docs look like at once.

Just add this to sphinx conf.py:

>>> html_css_files = ["https://malachov.github.io/readthedocs-sphinx-alabaster-css/custom.css"]

Also of course if you want you can download it and use locally from project if you need.

Result should look like this

sphinx-alabaster-css

Tasks

There are VS Code tasks examples in utils and build module, but here is small tutorial how to use it. Run command Tasks: Open User Tasks, add tasks from github/content/tasks or if have no task yet, you can copy / paste all.

Install extension Task Explorer

On root copy folder utils from content/tasks

You are ready to go. You should see something like this

tasks

You can do CI / CD pipeline or Build app with one click now.

Submodules

build

This module build the app via pyinstaller. It has presets to build applications build with eel.

There is one main function build_app. Check it’s help for how to use it (should be very simple).

Note

You can run build for example from vs code tasks, create folder utils, create build_script.py inside, add

>>> import mypythontools
...
>>> if __name__ == "__main__":
...     mypythontools.build.build_app()  # With all the params you need.

Then just add this task to global tasks.json:

{
    "label": "Build app",
    "type": "shell",
    "command": "python",
    "args": ["${workspaceFolder}/utils/build_script.py"],
    "presentation": {
        "reveal": "always",
        "panel": "new"
    }
},

config

This is not module that configure library mypythontools, but module that help create config for your project.

What

  1. Simple and short syntax.

  2. Ability to have docstrings on variables (not dynamically, so visible in IDE) and good for sphinx docs.

  3. Type checking, one-of-options checking.

  4. Also function evaluation from other config values (not only static value stored).

  5. Options hierarchy (nested options).

How

Boosted property with simplified implementation. To be able to provide docstrings in IDE, first function is created.

Examples:

>>> class SimpleConfig(ConfigBase):
...     @MyProperty(int)  # Use tuple like (str, int, bool) if more classes.
...     def var() -> int:  # This is for type hints in IDE.
...         '''
...         Type:
...             int
...
...         Default:
...             123
...
...         This is docstrings (also visible in IDE, because not defined dynamically).'''
...
...         return 123  # This is initial value that can be edited.
...
...     @MyProperty()  # Even if you don't need any params, use empty brackets
...     def evaluated(self) -> int:
...         return self.var + 1
...
>>> config = SimpleConfig()
>>> config.var
123
>>> config.var = 665
>>> config.var
665
>>> config['var']  # You can also access params as in a dictionary
665
>>> config.evaluated
666
>>> config.var = "String is problem"
Traceback (most recent call last):
TypeError: ...

This is how help looks like in VS Code

tasks

You can also use options checker validation.

>>> class SimpleConfig(ConfigBase):
...     @MyProperty(options=[1, 2, 3])
...     def var(self) -> int:
...         2  # This means that value will not be set on init
...
>>> config = SimpleConfig()
>>> config.var = 4
Traceback (most recent call last):
KeyError: ...

You can also edit getter or setter. You can use lambda function (if more logic, it’s better to use normal property and add type checking manually).

Use setattr and name with underscore as prefix or self.private_name self means property, not config object.

>>> class SimpleConfig(ConfigBase):
...     @MyProperty(
...         int,
...         fset=lambda self, object, new: (
...             print("I'm listener, i can call any function on change."),
...             setattr(object, self.private_name, new + 1),
...         ),
...     )
...     def with_setter() -> int:
...         return 1
...
>>> config = SimpleConfig()
>>> config.with_setter
1
>>> config.with_setter = 665
I'm listener, i can call any function on change.
>>> config.with_setter
666
Hierarchical config

If last level, do inherit from ConfigBase, else inherit from ConfigStructured

Note

Use unique values for all config variables even if they are in various subconfig.

>>> class Config(ConfigStructured):
...     def __init__(self) -> None:
...         self.subconfig1 = self.SubConfiguration1()
...         self.subconfig2 = self.SubConfiguration2()
...
...     class SubConfiguration1(ConfigStructured):
...         def __init__(self) -> None:
...             self.subsubconfig = self.SubSubConfiguration()
...
...         class SubSubConfiguration(ConfigBase):
...             @MyProperty(options=[0, 1, 2, 3])
...             def value1() -> int:
...                 '''Documentation here
...
...                 Options: [0, 1, 2, 3]
...                 '''
...                 return 3
...
...             @MyProperty()
...             def value2(self):
...                 return self.value1 + 1
...
...     class SubConfiguration2(ConfigBase):
...         @MyProperty()
...         def other_val(self):
...             return self.value2 + 1
...
...     # Also subconfig category can contain values itself
...     @MyProperty(options=[0, 1, 2, 3])
...     def value3() -> int:
...         return 3
...
>>> config = Config()
...
>>> config.subconfig1.subsubconfig.value2
4

You can access value from config as well as from subcategory

>>> config.value2
4
Copy

Sometimes you need more instances of settings and you need copy of existing configuration. Copy is deepcopy by default.

>>> config2 = config.copy()
>>> config2.value3 = 0
>>> config2.value3
0
>>> config.value3
3
Bulk update

Sometimes you want to update many values with flat dictionary.

>>> config.update({'value3': 2, 'value1': 0})
>>> config.value3
2
Get flat dictionary

There is a function that will export all the values to the flat dictionary (no dynamic anymore, just values).

>>> config.get_dict()
{'value3': 2, 'value1': 0, 'value2': 1, 'other_val': 2}
Reset

You can reset to default values

>>> config.value1 = 1
>>> config.reset()
>>> config.value1
3

Sphinx docs

If you want to have documentation via sphinx, you can add this to conf.py:

napoleon_custom_sections = [
    ("Types", "returns_style"),
    ("Type", "returns_style"),
    ("Options", "returns_style"),
    ("Default", "returns_style"),
    ("Example", "returns_style"),
    ("Examples", "returns_style"),
]

Here is example

tasks

deploy

Allow to deploy project. Possible destinations: PYPI.

Check deploy_to_pypi function docs for how to use it.

Usually this function is not called manually, but it’s a part of push_pipeline from utils.

Check utils docs where is described, how to use VS Code Task to be able to optionally test, push and deploy with tasks (one button click).

misc

Module with miscellaneous functions. For example myproperty, which is decarator for creating simplified properties or json_to_py that can convert json string to correct python types or str_to_infer_type that will convert string to correct type.

paths

Module where you can configure and process paths.

plots

Plot data. There is only one main function plot. Check it’s documentation for how to use it.

property

Module contains MyProperty class that edit normal python property to add new features.

There is default setter, it’s possible to auto init values on class init and values in setter can be validated.

It’s possible to set function as a value and it’s evaluated during call.

Example of how can it be used is in module config.

Examples:

>>> class MyClass:
...     # Init all default values of defined properties
...     def __init__(self):
...        for j in vars(type(self)).values():
...            if type(j) is MyProperty:
...                setattr(self, j.private_name, j.init_function)
...
...     @MyProperty(int)  # New value will be validated whether it's int
...     def var() -> int:  # This is for type hints in IDE.
...         '''
...         Type:
...             int
...
...         Default:
...             123
...
...         This is docstrings (also visible in IDE, because not defined dynamically).'''
...
...         return 123  # This is initial value that can be edited.
...
...     @MyProperty()  # Even if you don't need any params, use empty brackets
...     def var2():
...         return 111
...
>>> myobject = MyClass()
>>> myobject.var
123
>>> myobject.var = 124
>>> myobject.var
124

pyvueeel

Common functions for Python / Vue / Eel project.

It contains functions for running eel, overriding eel.expose decorator, converting json to correct python format or transform data into form for vue tables and plots.

Go on

https://mypythontools.readthedocs.io/#project-starter

for example with working examples.

Image of such an app

project-starter-gui

utils

This module can be used for example in running deploy pipelines or githooks (some code automatically executed before commit). This module can run the tests, edit library version, generate rst files for docs, push to git or deploy app to pypi.

All of that can be done with one function call - with push_pipeline function that run other functions, or you can use functions separately. If you are using other function than push_pipeline, you need to call mypythontools.paths.set_paths() first.

Examples:

VS Code Task example

You can push changes with single click with all the hooks displaying results in your terminal. All params changing every push (like git message or tag) can be configured on the beginning and therefore you don’t need to wait for test finish. Default values can be also used, so in small starting projects, push is actually very fast.

Create folder utils, create push_script.py inside, add

>>> import mypythontools
...
>>> if __name__ == "__main__":
...     # Params that are always the same define here. Params that are changing define in IDE when run action.
...     # For example in tasks (command line arguments and argparse will be used).
...     mypythontools.utils.push_pipeline(deploy=True)

Then just add this task to global tasks.json:

{
"version": "2.0.0",
"tasks": [
    {
    "label": "Build app",
    "type": "shell",
    "command": "python",
    "args": ["${workspaceFolder}/utils/build_script.py"],
    "presentation": {
        "reveal": "always",
        "panel": "new"
    }
    },
    {
    "label": "Hooks & push & deploy",
    "type": "shell",
    "command": "python",
    "args": [
        "${workspaceFolder}/utils/push_script.py",
        "--version",
        "${input:version}",
        "--commit_message",
        "${input:commit_message}",
        "--tag",
        "${input:tag}",
        "--tag_mesage",
        "${input:tag-message}"
    ],
    "presentation": {
        "reveal": "always",
        "panel": "new"
    }
    }
],
"inputs": [
    {
    "type": "promptString",
    "id": "version",
    "description": "Version in __init__.py will be overwritten. Version has to be in format like '1.0.3' three digits and two dots. If None, nothing will happen. If 'increment', than it will be updated by 0.0.1.",
    "default": "increment"
    },
    {
    "type": "promptString",
    "id": "commit_message",
    "description": "Git message for commit.",
    "default": "New commit"
    },
    {
    "type": "promptString",
    "id": "tag",
    "description": "Git tag. If '__version__' is used, then tag from version in __init__.py will be derived. E.g. 'v1.0.1' from '1.0.1'",
    "default": "__version__"
    },
    {
    "type": "promptString",
    "id": "tag-message",
    "description": "Git tag message.",
    "default": "New version"
    }
]
}

Git hooks example

Create folder git_hooks with git hook file - for prec commit name must be pre-commit (with no extension). Hooks in git folder are gitignored by default (and hooks is not visible on first sight).

Then add hook to git settings - run in terminal (last arg is path (created folder)):

$ git config core.hooksPath git_hooks

In created folder on first two lines copy this:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

Then just import any function from here and call with desired params. E.g.

>>> import mypythontools
>>> mypythontools.paths.set_paths()
>>> version = mypythontools.utils.get_version()  # For example 1.0.2
>>> print(version[0].isdigit() == version[2].isdigit() == version[5].isdigit() == (version[1] == '.' == version[3]))
True