Building a System Root Directory¶
pyqtdeploy-sysroot is used to create a target-specific system root directory (sysroot) containing the target Python installation and any third-party components required by the application. It’s use is optional but highly recommended.
pyqtdeploy-sysroot is actually a wrapper around a number of component plugins. A plugin, written in Python, is responsible for building and/or installing (in the sysroot) an individual component. It would normally be a Python package or extension module but could just as easily be a supporting library.
A sysroot is defined by a JSON sysroot specification file. This contains an object for each component to build and/or install. The attributes of an object determine how the component is configured. Component and attribute names may be scoped in the same way as qmake variables (described in Adding Other Extension Modules) so that components can be included and configured on a target by target basis.
The components are built and/or installed in the order in which their objects appear in the specification file.
An API is provided to allow you to develop your own component plugins. If you develop a plugin for a commonly used component then please consider contributing it so that it can be included in a future release of pyqtdeploy.
Standard Component Plugins¶
The following component plugins are included as standard with pyqtdeploy.
- openssl
- This builds the OpenSSL libraries for v1.0.* and v1.1.* on Android
(dynamically), macOS (statically) and Windows (statically). It requires
perl
to be installed onPATH
. - pip
- This is a meta-component which will install any number of components that
can be installed by
pip
. - pyqt3d
- This builds a static version of the PyQt3D extension modules for all target architectures. It must be built after PyQt5.
- pyqt5
- This builds a static version of the PyQt5 extension modules for all target architectures. It must be built after sip and Qt5.
- pyqtchart
- This builds a static version of the PyQtChart extension module for all target architectures. It must be built after PyQt5.
- pyqtdatavisualization
- This builds a static version of the PyQtDataVisualization extension module for all target architectures. It must be built after PyQt5.
- pyqtpurchasing
- This builds a static version of the PyQtPurchasing extension module for all target architectures. It must be built after PyQt5.
- python
- This will build Python from source, or install it into sysroot from an existing installation, for both the host and target architectures. Building the host version from source is not supported on Windows. Installing the host version from an existing installation is not supported on Android or iOS. The target version of the Python library and extension modules built from source will be built statically. Installing the target version from an existing installation is only supported on Windows. If building the target version from source and SSL support is required then OpenSSL must be built first. If building Python v3.7.0 or later and zlib support is needed then zlib must be built first.
- qscintilla
- This builds a static version of the QScintilla library and Python extension module for all target architectures. It must be built after PyQt5.
- qt5
- This will build a static version of Qt5 from source (but not for the
Android and iOS targets). It will also install Qt5 into the sysroot from
an existing installation created by the standard installer. When building
from source on Windows it requires Python v2.7 to be installed (but it does
not need to be on
PATH
). If building from source and SSL support using OpenSSL is required then OpenSSL must be built first. - sip
- This builds a static version of the sip extension module for all target architectures. It also builds the sip code generator for the host platform. It must be built after Python.
- zlib
- This builds a static version of the zlib library for all target architectures.
Creating a Sysroot Specification File¶
The following specification file contains an object for each of the standard
component plugins. (You can also download a copy of the file from
here
). No configuration options have been
set for any component.
{
"Description": "A skeleton sysroot specification.",
"openssl": {
},
"zlib": {
},
"qt5": {
},
"python": {
},
"sip": {
},
"pyqt5": {
},
"pyqt3d": {
},
"pyqtchart": {
},
"pyqtdatavisualization": {
},
"pyqtpurchasing": {
},
"qscintilla": {
},
"pip": {
}
}
The first object, called Description
, is simple a way of including a
comment in the specification and is otherwise ignored.
Using this file, run the following command:
pyqtdeploy-sysroot --options skeleton.json
You will then see a description of each component’s configuration options, the type of value expected and whether or not a value is required. You can then add options as attributes to the appropriate objects to meet your requirements.
If your application does not require all of the standard components then simply remove the corresponding objects from the specification file. If your application requires additional components then you need to create appropriate component plugins and add corresponding objects to the specification file.
To build a native sysroot (i.e. for the host architecture) from a fully configured specification file, run:
pyqtdeploy-sysroot skeleton.json
The pyqt-demo Sysroot¶
In this section we walk through the sysroot specification file for pyqt-demo, component by component.
openssl¶
"android|macos|win#openssl": {
"android#source": "openssl-1.0.*.tar.gz",
"macos|win#source": "openssl-1.1.*.tar.gz"
},
The first thing to notice is that the object name is scoped so that the component is only built for Android, macOS and Windows. On iOS we choose to not support SSL from Python and use Qt’s SSL support instead (which will use Apple’s Secure Transport). On Linux we will use the system versions of the OpenSSL libraries.
On Android we use OpenSSL v1.0 because that is the version used by the pre-built binaries provided by the Qt installer.
On macOS and Windows we choose to use OpenSSL v1.1.
zlib¶
"ios|linux|macos|win#zlib": {
"source": "zlib-*.tar.gz",
"static_msvc_runtime": true
},
On Android we are using the zlib library provided on the device. On other architectures we choose to use a static version of the library. On Windows we choose to link to static versions of the MSVC runtime libraries.
qt5¶
"qt5": {
"android#qt_dir": "android_armv7",
"ios#qt_dir": "ios",
"linux|macos|win#source": "qt-everywhere-*-src-5.*.tar.xz",
"android|linux#ssl": "openssl-runtime",
"ios#ssl": "securetransport",
"macos|win#ssl": "openssl-linked",
"configure_options": [
"-opengl", "desktop", "-no-dbus", "-qt-pcre"
],
"skip": [
"qtactiveqt", "qtconnectivity", "qtdoc", "qtgamepad",
"qtlocation", "qtmultimedia", "qtnetworkauth",
"qtquickcontrols", "qtquickcontrols2", "qtremoteobjects",
"qtscript", "qtscxml", "qtsensors", "qtserialbus",
"qtserialport", "qtspeech", "qtsvg", "qttools",
"qttranslations", "qtwayland", "qtwebchannel", "qtwebengine",
"qtwebsockets", "qtwebview", "qtxmlpatterns"
],
"static_msvc_runtime": true
},
The Qt5 component plugin will install Qt into the sysroot from an existing
installation. We have chosen to do this for Android and iOS by specifying the
qt_dir
attribute. In the context of the demo this is defined by the
--installed-qt-dir
option of the build-demo.py
script.
The plugin will build Qt from source if the source
attribute is specified.
We have chosen to do this for Linux, macOS and Windows.
The ssl
attribute specifies how Qt’s SSL support is to be implemented.
For Android and Linux we have chosen to dynamically load external OpenSSL
libraries at runtime. On Android the external libraries are those built by the
openssl
component plugin and are bundled automaticallly with the
application executable. On Linux the external libraries are the system
versions.
For iOS Qt is dynamically linked to the Secure Transport libraries.
For macOS and Windows we have chosen to link against the static OpenSSL
libraries built by the openssl
component plugin.
Next we use the configure_options
and skip
attributes to tailor the Qt
build in order to reduce the time taken to do the build.
Finally we have specified that (on Windows) we will link to static versions of the MSVC runtime libraries.
python¶
"python": {
"build_host_from_source": false,
"build_target_from_source": true,
"source": "Python-3.7.0.tar.xz"
},
The Python component plugin handles installation for both host and target
architectures. For the host we choose to use an existing Python installation
rather than build from source. On Windows the registry is searched for the
location of the existing installation. On Linux and macOS the Python
interpreter must be on PATH
. For all target architecures we choose
to build Python from source.
We have chosen to be very specific about the version number of Python (rather
than use a pattern) because it must match the version specified in the
pyqt-deploy.pdy
project file.
pyqt-demo is a very simple application that does not need to
dynamically load extension modules. If this was needed then the
dynamic_loading
attribute would be set to true
.
sip¶
"sip": {
"source": "sip-4.*.tar.gz"
},
It is only necessary to specifiy the name of the source archive.
pyqt5¶
"pyqt5": {
"android#disabled_features": [
"PyQt_Desktop_OpenGL", "PyQt_Printer", "PyQt_PrintDialog",
"PyQt_PrintPreviewDialog", "PyQt_PrintPreviewWidget"
],
"android#modules": [
"QtCore", "QtGui", "QtNetwork", "QtPrintSupport", "QtWidgets",
"QtAndroidExtras"
],
"ios#disabled_features": [
"PyQt_Desktop_OpenGL", "PyQt_MacOSXOnly",
"PyQt_MacCocoaViewContainer", "PyQt_Printer",
"PyQt_PrintDialog", "PyQt_PrintPreviewDialog",
"PyQt_PrintPreviewWidget", "PyQt_Process",
"PyQt_NotBootstrapped"
],
"ios|macos#modules": [
"QtCore", "QtGui", "QtNetwork", "QtPrintSupport", "QtWidgets",
"QtMacExtras"
],
"linux#modules": [
"QtCore", "QtGui", "QtNetwork", "QtPrintSupport", "QtWidgets",
"QtX11Extras"
],
"win#disabled_features": ["PyQt_Desktop_OpenGL"],
"win#modules": [
"QtCore", "QtGui", "QtNetwork", "QtPrintSupport", "QtWidgets",
"QtWinExtras"
],
"source": "PyQt5_*-5.*.tar.gz"
},
The two attributes used to tailor the build of PyQt5 are disabled_features
and modules
.
Unfortunately the list of features that can be disabled is not properly
documented and relate to how Qt5 was configured. However how
disabled_features
is set in the above will be appropriate for most cases.
The modules
attribute is used to specify the names of the individual PyQt
extension modules to be built. We choose to build only those extension
modules needed by pyqt-demo.
pyqt3D¶
"pyqt3d": {
"source": "PyQt3D_*-5.*.tar.gz"
},
It is only necessary to specifiy the name of the source archive.
pyqtchart¶
"pyqtchart": {
"source": "PyQtChart_*-5.*.tar.gz"
},
It is only necessary to specifiy the name of the source archive.
pyqtdatavisualization¶
"pyqtdatavisualization": {
"source": "PyQtDataVisualization_*-5.*.tar.gz"
},
It is only necessary to specifiy the name of the source archive.
pyqtpurchasing¶
"pyqtpurchasing": {
"source": "PyQtPurchasing_*-5.*.tar.gz"
},
It is only necessary to specifiy the name of the source archive.
qscintilla¶
"qscintilla": {
"source": "QScintilla_*-2.*.tar.gz"
}
It is only necessary to specifiy the name of the source archive.
The Command Line¶
The full set of command line options is:
-
-h
,
--help
¶
This will display a summary of the command line options.
-
--component
COMPONENT
¶ COMPONENT
is the name of the component (specified in the JSON file) that will be built. It may be used more than once to build multiple components. If the option is not specified then all components specified in the JSON file will be built.
-
--no-clean
¶
A temporary build directory (called
build
in the sysroot) is created in order to build the required components. Normally this is removed automatically after all components have been built. Specifying this option leaves the build directory in place to make debugging component plugins easier.
-
--options
¶
This causes the configurable options of each component specified in the JSON file to be displayed on
stdout
. The program will then terminate.
-
--plugin-dir
DIR
¶ DIR
is added to the list of directories that are searched for component plugins. It may be used more than once to search multiple directories. All directories specified in this way will be searched before those directories (internal to pyqtdeploy-sysroot) that are searched by default.
-
--source-dir
DIR
¶ DIR
is the name of a directory containing the source archives used to build the components specified in the JSON file. It may be specified any number of times and each directory will be searched in turn. If it is omitted then the current directory is searched.
-
--sysroot
DIR
¶ DIR
is the name of the system root directory. The default value issysroot-
followed by a target-specific suffix. Unless the--component
option is specified any existing sysroot will first be removed and re-created.
-
--target
TARGET
¶ TARGET
is the target architecture. By default the host architecture is used.
-
--quiet
¶
This specifies that progress messages should be disabled.
-
--verbose
¶
This specifies that additional progress messages should be enabled.
-
-V
,
--version
¶
This specifies that the version number should be displayed on
stdout
. The program will then terminate.
-
specification
¶
specification
is the name of a JSON specification file that defines each component to be included in the sysroot and how each is to be configured.
Writing a Component Plugin¶
A component plugin is a Python module that defines a sub-class of
pyqtdeploy.ComponentBase
. The sub-class must re-implement the
build()
method and may also
re-implement the configure()
method. It
should also include a class attribute called
options
which is a sequence of
pyqtdeploy.ComponentOption
instances that describe each of the
component’s configurable options. It does not matter what the name of the
class is.
-
class
pyqtdeploy.
ComponentBase
¶ This is the base class of all component plugins.
-
options
¶ This class attribute is a sequence of
ComponentOption
instances describing the component’s configurable options.
-
-
class
pyqtdeploy.
ComponentOption
(name, type=str, required=False, default=None, values=None, help='')¶ This class implements an option used to configure the component. An option can be specified as an attribute of the component’s object in the sysroot specification file. An instance of the component plugin will contain an attribute for each option whose value is that specified in the sysroot specification file (or an appropriate default if it was omitted).
Parameters: - name (str) – the name of the option.
- type (
bool
,int
,list
orstr
) – the type of a value of the option. - required (bool) –
True
if a value for the option is required. - default – the default value of the option.
- values – the possible values of the option.
- help (str) – the help text displayed by the
--options
option of pyqtdeploy-sysroot.
-
class
pyqtdeploy.
Sysroot
¶ This class encapsulates a sysroot as seen by a component plugin. Instances are only created by pyqtdeploy-sysroot and are passed to the plugin when required.
-
add_to_path
(name)¶ The name of a directory is added to the start of
PATH
if it isn’t already present.Parameters: name (str) – is the name of the directory. Returns: the original value of PATH
.
-
android_api
¶ The numerical Android API level to use.
-
android_ndk_sysroot
¶ The path of the Android NDK’s sysroot directory.
-
android_toolchain_bin
¶ The path of the Android toolchain’s bin directory.
-
android_toolchain_prefix
¶ The name of the Android toolchain’s prefix.
-
apple_sdk
¶ The Apple SDK to use.
-
building_for_target
¶ This is set to
True
by the component plugin to configure building (i.e. compiling and linking) for the target (rather than the host) architecture. The default value isTrue
.
-
components
¶ The sequence of component names in the sysroot specification.
-
copy_file
(src, dst)¶ A file is copied. Any errors are handled automatically.
Parameters: - src (str) – is the name of the source file.
- dst (str) – is the name of the destination file.
-
copy_dir
(src, dst, ignore=None)¶ A directory is copied, optionally excluding file and sub-directories that match a number of glob patterns. If the destination directory already exists then it is first removed. Any errors are handled automatically.
Parameters: - src (str) – is the name of the source directory.
- dst (str) – is the name of the destination directory.
- ignore (list[str]) – is an optional sequence of glob patterns that specify files and sub-directories that should be ignored.
-
create_file
(name)¶ A new text file is created and its file object returned. Any errors are handled automatically.
Parameters: name (str) – is the name of the file. Returns: the file object of the created file.
-
create_dir
(name, empty=False)¶ A new directory is created if it does not already exist. If it does already exist then it is optionally emptied. Any errors are handled automatically.
Parameters: - name (str) – is the name of the directory.
- empty (bool) –
True
if an existing directory should be emptied.
-
decode_version_nr
(version_nr)¶ An encoded version number is decoded to a 3-tuple of major version, minor version and patch version.
Parameters: version_nr (int) – is the encoded version number. Returns: the decoded 3-tuple.
-
delete_dir
(name)¶ A directory and any contents are deleted. Any errors are handled automatically.
Parameters: name (str) – is the name of the directory.
-
error
(message, detail='')¶ An error message is displayed to the user and the program immediately terminates.
Parameters: - message (str) – is the message.
- detail (str) – is additional detail displayed if the
--verbose
option was specified.
-
extract_version_nr
(name)¶ An encoded version number is extracted from the name of file or directory based on common naming standards.
Parameters: name (str) – is the name of the file or directory. Returns: the encoded version number.
-
find_component
(name, required=True)¶ The
ComponentBase
instance for a component is returned.Parameters: - name (str) – is the name of the component.
- required (bool) –
True
if the component must exist.
Returns: the component instance.
-
find_exe
(name)¶ The absolute path name of an executable located on
PATH
is returned. Any errors are handled automatically.Parameters: name (str) – is the generic executable name. Returns: the absolute path name of the executable.
-
find_file
(name, required=True)¶ The absolute path name of a file or directory is returned. If the name is relative then it is assumed t be relative to the directory specified by the
--source-dir
option. If this option has not been specified then the directory containing the JSON specification file is used. The name may be a glob pattern but must only identify a single file or directory.Parameters: - name (str) – is the name of the file or directory.
- required (bool) –
True
if the file or directory must exist.
Returns: the absolute path name of the file or directory.
-
format_version_nr
(version_nr)¶ An encoded version number is converted to a string.
Parameters: version_nr (int) – is the encoded version number. Returns: the string conversion.
-
get_python_install_path
(version_nr=None)¶ The name of the directory containing the root of a Python installation on Windows is returned. If an encoded version number is not given then
target_py_version_nr
is used. It must only be called by a Windows host.Parameters: version_nr (int) – is the encoded version number. Returns: the absolute path of the installation directory.
-
host_arch_name
¶ The name of the host architecture.
-
host_bin_dir
¶ The name of the directory where executables built for the host architecture should be installed.
-
host_dir
¶ The name of the root directory where components built for the host architecture should be installed.
-
host_exe
(name)¶ A generic executable name is converted to a host-specific version.
Parameters: name (str) – is the generic name. Returns: the host-specific name.
-
host_make
¶ The name of the host
make
executable.
-
host_platform_name
¶ The name of the host platform.
-
host_pip
¶ The full path name of the host
pip
executable.
-
host_python
¶ The full path name of the host
python
executable.
-
host_qmake
¶ The full path name of the host
qmake
executable.
-
host_sip
¶ The full path name of the host
sip
executable.
-
make_symlink
(src, dst)¶ A symbolic link is made between source and destination files. (Note that, on Windows, a copy of the source file is made.)
Parameters: - src (str) – is the name of the source file.
- dst (str) – is the name of the destination link that is created.
-
open_file
(name)¶ An existing text file is opened and its file object returned. Any errors are handled automatically.
Parameters: name (str) – is the name of the file. Returns: the file object of the opened file.
-
parse_version_nr
(version_str)¶ Convert a string in the form [M[.m[.p]]] to an encoded version number.
Parameters: version_str (str) – is the version number to parse. Returns: an encoded version number.
-
pip_install
(package)¶ Install a package using
host_pip
totarget_sitepackages_dir
. The package may refer either to a local file or to one to be downloaded from PyPI.Parameters: package (str) – is the name of the package.
-
progress
(message)¶ A progress message is displayed to the user. It will be suppressed if the
--quiet
option was specified.Parameters: message (str) – is the message.
-
run
(*args, capture=False)¶ An external command is run. The command’s stdout can be optionally captured.
Parameters: - *args – are the name of the command and its arguments.
- capture (bool) –
True
if the command’s stdout should be captured and returned.
Returns: the stdout of the command if requested, otherwise
None
.
-
target_arch_name
¶ The name of the target architecture.
-
target_include_dir
¶ The name of the directory where header files built for the target architecture should be installed.
-
target_lib_dir
¶ The name of the directory where libraries built for the target architecture should be installed.
-
target_platform_name
¶ The name of the target platform.
-
target_py_include_dir
¶ The name of the directory where
Python.h
built for the target architecture can be found.
-
target_py_lib
¶ The name of Python library built for the target architecture.
-
target_py_stdlib_dir
¶ The name of the directory where the Python standard library built for the target architecture can be found.
-
target_py_version_nr
¶ The version of Python being targeted.
-
target_pyqt_platform
¶ The name of the target platform as known by PyQt’s
configure.py
.
-
target_sip_dir
¶ The name of the directory where
.sip
files built for the target architecture can be found.
-
target_sitepackages_dir
¶ The name of the
site-packages
directory for the target architecture.
-
target_src_dir
¶ The name of the directory where source files can be found. Note that these are sources left by components for the use of other components and not the sources used to build a component.
-
unpack_archive
(archive, chdir=True)¶ An archive (e.g. a
.tar.gz
or.zip
file) is unpacked in the current directory.Parameters: - archive (str) – the name of the archive.
- chdir (bool) –
True
if the top level directory of the extracted archive should become the new current directory.
Returns: the name of the top level directory of the extracted archive excluding any path.
-