Some tools/functions/snippets/files used across projects.
Can be used as python library. Things like building the application with pyinstaller,
incrementing version,
generating rst files for sphinx docs, pushing to github or deploying to PyPi or other CI/CD
functionality,
creating config module or plot data is matter of calling one function or clicking one button
(e.g. Vs code task).
Many projects - one codebase.
If you are not sure whether structure of your app will work with this code, check
project-starter on github
in content folder.
Paths are inferred, but if you have atypical structure or have more projects in cwd, use
mypythontools.paths.set_paths() with path arguments.
There is also some extra stuff, that is not bundled via PyPI (cookiecutter, CSS for
readthedocs etc.),
such a content is under the Content topic.
Content
There are a lot of stuff that’s not in python library (installable via pip), but still on
github repository.
Link where you can find find that content:
https://github.com/Malachov/mypythontools/tree/master/content
Link where you can read about how to use it:
https://mypythontools.readthedocs.io/#content
Some examples of what you can find only on github
project-starter
Project scaffolding fast and easy.
Download project-starter from
https://github.com/Malachov/mypythontools/tree/master/content/project-starter
(You may need to download all mypythontools repository in zip)
And start developing.
In your IDE do bulk renaming across files and replace SET_YOUR_APP_NAME
with name of your app / library.
This starter is for vue-eel applications (desktop as well as web) but also for python
libraries that will be
stored on PyPi.
If it’s python library, delete gui folder.
If it’s app with gui, delete setup.py, __init__.py and
remove Installation and Modules from
README. Also from assets delete mdi.css if not using icons and
formulate.css if not using vue
formulate. In package.json uncomment library you will be
using.
Install used python libraries via pip install -r requirements.txt and
install JS libraries as well with
npm install in folder where package.json is.
To run an app in develop mode, you have to run both frontent as well as python. Run
frontent with debugging
app.py (do not run, just debug). Then run frontend with npm run serve
in gui folder (or use Task explorer if
using VS Code). Open your favourite browser and open
http://localhost:8080
It’s recommended to use Vue.js devtools extension where you can see what component is
on cursor, edit props
values or see list of all used mutations.
In opened app, there is a little help button where there is simple overview about how
to develop with these
tools.
Delete is faster than write, so there is many working examples like for example plot,
various formating (flex
row, flex column), settings panel, function call from python to js and vice versa or
automatic alerting from
python. If you want to see how some example is working, just use ctrl +
F in IDE or check components for
it’s props.
This is how the example looks like
Docs
It includes docs structure for sphinx docs generations from docstrings. Edit first
few lines in conf.py,
index.rst, navi.html and if you want, add static files like logo to
_static.
Usually used with https://readthedocs.org/ free hosting that
trigger deploy automatically after pushing to
master. Because of correct relative redirecting in index.rst and navi.html use for
readthedocs /path/ before
relative link to ohter module.
It’s also necassary to generate other modules rst files for other pages but it’s if
using push_script.py as
CI/CD (see below), it’s generated automatically via apidoc.
It’s recommended to use with sphinx-alabaster-css (see below).
CI/CD
It also include github actions yml file for pushing codecov coverage file (without
token) and .travis.yml for
Travis CI. You don’t need those files and can you can delete it.
There is file push_script.py in folder utils which
personally is better (faster) than travis and can do
most of what you want from CI like run pipeline - running tests / generate docs /
increment version / push to
github / push to pypi.
Check utils module for more informations.
IDE files
- It also include some default project specific
-
settings for VS Code. You can also delete it.
If developing py - vue - eel app this is the recommended way, but if you want to
build just what is necessary
from scratch, you can use this tutorial
https://mypythontools.readthedocs.io/mypythontools.pyvueeel.html#mypythontools.pyvueeel.help_starter_pack_vue_app
requirements
Install many libraries at once (no need for Anaconda). Download
requirements.txt file from
https://github.com/Malachov/mypythontools/tree/master/content/requirements
and in that folder use:
pip install -r requirements.txt
It’s good for python libraries that other users with different versions of libraries
will use. If not
standalone application where freezing into virtual env is good idea - here is
possible to use this
requirements with using –upgrade from time to time to be sure that your library will
be working for up to
date version of dependencies.
sphinx-alabaster-css
It’s good idea to generate documentation from code. If you are useing sphinx and
alabaster theme, you can use
this css file for formating.
Tested on readthedocs hosting (recommended).
CSS are served from github and it’s possible to change on one place and edit how all
projects docs look like
at once.
Just add this to sphinx conf.py:
>>> html_css_files = ["https://malachov.github.io/readthedocs-sphinx-alabaster-css/custom.css"]
Also of course if you want you can download it and use locally from project if you
need.
Result should look like this
Tasks
There are VS Code tasks examples in utils and build module, but here is small
tutorial how to use it. Run
command Tasks: Open User Tasks, add tasks from github/content/tasks or
if have no task yet, you can copy /
paste all.
Install extension Task Explorer
On root copy folder utils from content/tasks
You are ready to go. You should see something like this
You can do CI / CD pipeline or Build app with one click now.
Submodules
This module build the app via pyinstaller.
It has presets to build applications build with eel.
There is one main function build_app. Check it’s help for how to use
it (should be very simple).
Note
You can run build for example from vs code tasks, create folder utils,
create build_script.py inside, add
>>> import mypythontools
...
>>> if __name__ == "__main__":
... mypythontools.build.build_app() # With all the params you need.
Then just add this task to global tasks.json:
{
"label": "Build app",
"type": "shell",
"command": "python",
"args": ["${workspaceFolder}/utils/build_script.py"],
"presentation": {
"reveal": "always",
"panel": "new"
}
},
This is not module that configure library mypythontools, but module that help create
config for your project.
What
-
Simple and short syntax.
-
Ability to have docstrings on variables (not dynamically, so visible in
IDE) and good for sphinx docs.
-
Type checking, one-of-options checking.
-
Also function evaluation from other config values (not only static value
stored).
-
Options hierarchy (nested options).
How
Boosted property with simplified implementation. To be able to provide docstrings
in IDE, first function is created.
Examples:
>>> class SimpleConfig(ConfigBase):
... @MyProperty(int) # Use tuple like (str, int, bool) if more classes.
... def var() -> int: # This is for type hints in IDE.
... '''
... Type:
... int
...
... Default:
... 123
...
... This is docstrings (also visible in IDE, because not defined dynamically).'''
...
... return 123 # This is initial value that can be edited.
...
... @MyProperty() # Even if you don't need any params, use empty brackets
... def evaluated(self) -> int:
... return self.var + 1
...
>>> config = SimpleConfig()
>>> config.var
123
>>> config.var = 665
>>> config.var
665
>>> config['var'] # You can also access params as in a dictionary
665
>>> config.evaluated
666
>>> config.var = "String is problem"
Traceback (most recent call last):
TypeError: ...
This is how help looks like in VS Code
You can also use options checker validation.
>>> class SimpleConfig(ConfigBase):
... @MyProperty(options=[1, 2, 3])
... def var(self) -> int:
... 2 # This means that value will not be set on init
...
>>> config = SimpleConfig()
>>> config.var = 4
Traceback (most recent call last):
KeyError: ...
You can also edit getter or setter. You can use lambda function (if more logic,
it’s better to use normal property and add type checking manually).
Use setattr and name with underscore as prefix or self.private_name self means
property, not config object.
>>> class SimpleConfig(ConfigBase):
... @MyProperty(
... int,
... fset=lambda self, object, new: (
... print("I'm listener, i can call any function on change."),
... setattr(object, self.private_name, new + 1),
... ),
... )
... def with_setter() -> int:
... return 1
...
>>> config = SimpleConfig()
>>> config.with_setter
1
>>> config.with_setter = 665
I'm listener, i can call any function on change.
>>> config.with_setter
666
Hierarchical config
If last level, do inherit from ConfigBase, else inherit from ConfigStructured
Note
Use unique values for all config variables even if they are in various
subconfig.
>>> class Config(ConfigStructured):
... def __init__(self) -> None:
... self.subconfig1 = self.SubConfiguration1()
... self.subconfig2 = self.SubConfiguration2()
...
... class SubConfiguration1(ConfigStructured):
... def __init__(self) -> None:
... self.subsubconfig = self.SubSubConfiguration()
...
... class SubSubConfiguration(ConfigBase):
... @MyProperty(options=[0, 1, 2, 3])
... def value1() -> int:
... '''Documentation here
...
... Options: [0, 1, 2, 3]
... '''
... return 3
...
... @MyProperty()
... def value2(self):
... return self.value1 + 1
...
... class SubConfiguration2(ConfigBase):
... @MyProperty()
... def other_val(self):
... return self.value2 + 1
...
... # Also subconfig category can contain values itself
... @MyProperty(options=[0, 1, 2, 3])
... def value3() -> int:
... return 3
...
>>> config = Config()
...
>>> config.subconfig1.subsubconfig.value2
4
You can access value from config as well as from subcategory
Copy
Sometimes you need more instances of settings and you need copy of existing
configuration. Copy is deepcopy by default.
>>> config2 = config.copy()
>>> config2.value3 = 0
>>> config2.value3
0
>>> config.value3
3
Bulk update
Sometimes you want to update many values with flat dictionary.
>>> config.update({'value3': 2, 'value1': 0})
>>> config.value3
2
Get flat dictionary
There is a function that will export all the values to the flat dictionary
(no dynamic anymore, just values).
>>> config.get_dict()
{'value3': 2, 'value1': 0, 'value2': 1, 'other_val': 2}
Reset
You can reset to default values
>>> config.value1 = 1
>>> config.reset()
>>> config.value1
3
Sphinx docs
If you want to have documentation via sphinx, you can add this to conf.py:
napoleon_custom_sections = [
("Types", "returns_style"),
("Type", "returns_style"),
("Options", "returns_style"),
("Default", "returns_style"),
("Example", "returns_style"),
("Examples", "returns_style"),
]
Here is example
Allow to deploy project.
Possible destinations: PYPI.
Check deploy_to_pypi function docs for how to use it.
Usually this function is not called manually, but it’s a part of
push_pipeline from utils.
Check utils docs where is described, how to use VS Code Task to be able to optionally
test, push and deploy
with tasks (one button click).
Module with miscellaneous functions. For example myproperty, which is decarator for
creating
simplified properties or json_to_py that can convert json string to correct python
types or
str_to_infer_type that will convert string to correct type.
Module where you can configure and process paths.
Plot data. There is only one main function plot. Check it’s documentation for how to
use it.
Module contains MyProperty class that edit normal python property to add new
features.
There is default setter, it’s possible to auto init values on class init and values
in setter can be
validated.
It’s possible to set function as a value and it’s evaluated during call.
Example of how can it be used is in module config.
Examples:
>>> class MyClass:
... # Init all default values of defined properties
... def __init__(self):
... for j in vars(type(self)).values():
... if type(j) is MyProperty:
... setattr(self, j.private_name, j.init_function)
...
... @MyProperty(int) # New value will be validated whether it's int
... def var() -> int: # This is for type hints in IDE.
... '''
... Type:
... int
...
... Default:
... 123
...
... This is docstrings (also visible in IDE, because not defined dynamically).'''
...
... return 123 # This is initial value that can be edited.
...
... @MyProperty() # Even if you don't need any params, use empty brackets
... def var2():
... return 111
...
>>> myobject = MyClass()
>>> myobject.var
123
>>> myobject.var = 124
>>> myobject.var
124
Common functions for Python / Vue / Eel project.
It contains functions for running eel, overriding eel.expose decorator, converting
json to correct python
format or transform data into form for vue tables and plots.
Go on
https://mypythontools.readthedocs.io/#project-starter
for example with working examples.
Image of such an app
This module can be used for example in running deploy pipelines or githooks
(some code automatically executed before commit). This module can run the tests,
edit library version, generate rst files for docs, push to git or deploy app to
pypi.
All of that can be done with one function call - with push_pipeline
function that
run other functions, or you can use functions separately. If you are using other
function than push_pipeline, you need to call
mypythontools.paths.set_paths() first.
Examples:
VS Code Task example
You can push changes with single click with all the hooks displaying
results in
your terminal. All params changing every push (like git message or tag)
can
be configured on the beginning and therefore you don’t need to wait for
test finish.
Default values can be also used, so in small starting projects, push is
actually very fast.
Create folder utils, create push_script.py inside, add
>>> import mypythontools
...
>>> if __name__ == "__main__":
... # Params that are always the same define here. Params that are changing define in IDE when run action.
... # For example in tasks (command line arguments and argparse will be used).
... mypythontools.utils.push_pipeline(deploy=True)
Then just add this task to global tasks.json:
{
"version": "2.0.0",
"tasks": [
{
"label": "Build app",
"type": "shell",
"command": "python",
"args": ["${workspaceFolder}/utils/build_script.py"],
"presentation": {
"reveal": "always",
"panel": "new"
}
},
{
"label": "Hooks & push & deploy",
"type": "shell",
"command": "python",
"args": [
"${workspaceFolder}/utils/push_script.py",
"--version",
"${input:version}",
"--commit_message",
"${input:commit_message}",
"--tag",
"${input:tag}",
"--tag_mesage",
"${input:tag-message}"
],
"presentation": {
"reveal": "always",
"panel": "new"
}
}
],
"inputs": [
{
"type": "promptString",
"id": "version",
"description": "Version in __init__.py will be overwritten. Version has to be in format like '1.0.3' three digits and two dots. If None, nothing will happen. If 'increment', than it will be updated by 0.0.1.",
"default": "increment"
},
{
"type": "promptString",
"id": "commit_message",
"description": "Git message for commit.",
"default": "New commit"
},
{
"type": "promptString",
"id": "tag",
"description": "Git tag. If '__version__' is used, then tag from version in __init__.py will be derived. E.g. 'v1.0.1' from '1.0.1'",
"default": "__version__"
},
{
"type": "promptString",
"id": "tag-message",
"description": "Git tag message.",
"default": "New version"
}
]
}
Git hooks example
Create folder git_hooks with git hook file - for prec commit name must be
pre-commit
(with no extension). Hooks in git folder are gitignored by default (and
hooks is not visible
on first sight).
Then add hook to git settings - run in terminal (last arg is path
(created folder)):
$ git config core.hooksPath git_hooks
In created folder on first two lines copy this:
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
Then just import any function from here and call with desired params.
E.g.
>>> import mypythontools
>>> mypythontools.paths.set_paths()
>>> version = mypythontools.utils.get_version() # For example 1.0.2
>>> print(version[0].isdigit() == version[2].isdigit() == version[5].isdigit() == (version[1] == '.' == version[3]))
True