cli provides a suite of tools that simplify the process of developing testable, robust and easy-to-use command line applications. With cli, your scripts (and larger applications) can use Python’s extensive standard library to parse configuration files and command line parameters, log messages and even daemonize. All of these services are made available through the cli.app.Application class, an instance of which is passed to your script at run time. Even though it is featureful, cli is decidedly unmagical.
You can install the latest stable version of cli using pip:
$ pip install pyCLI
You can also browse the project’s Mercurial repository. If you’d like to contribute changes, start by cloning the repository:
$ hg clone http://code.lfod.us/cli
Command line parsing:
#!/usr/bin/env python
import cli
@cli.CommandLineApp
def ls(app):
pass
ls.add_param("-l", "--long", help="list in long format", default=False, action="store_true")
if __name__ == "__main__":
ls.run()
When run, this script produces the following output:
$ python ls.py -h
usage: ls [-h] [-l]
optional arguments:
-h, --help show this help message and exit
-l, --long list in long format
Logging:
#!/usr/bin/env python
import time
import cli
@cli.LoggingApp
def sleep(app):
app.log.debug("About to sleep for %d seconds" % app.params.seconds)
time.sleep(app.params.seconds)
sleep.add_param("seconds", help="time to sleep", default=1, type=int)
if __name__ == "__main__":
sleep.run()
Which produces the following:
$ python sleep.py -h
usage: sleep [-h] [-l] [-q] [-s] [-v] seconds
positional arguments:
seconds time to sleep
optional arguments:
-h, --help show this help message and exit
-l, --logfile log to file (default: log to stdout)
-q, --quiet decrease the verbosity
-s, --silent only log warnings
-v, --verbose raise the verbosity
$ python sleep.py -vv 3
About to sleep for 3 seconds
Daemonizing:
#!/usr/bin/env python
import cli
@cli.DaemonizingApp
def daemon(app):
if app.params.daemonize:
app.log.info("About to daemonize")
app.daemonize()
if __name__ == "__main__":
daemon.run()
And on the command line:
$ python daemon.py -h
usage: daemon [-h] [-l] [-q] [-s] [-v] [-d] [-u USER] [-p PIDFILE]
optional arguments:
-h, --help show this help message and exit
-l, --logfile log to file (default: log to stdout)
-q, --quiet decrease the verbosity
-s, --silent only log warnings
-v, --verbose raise the verbosity
-d, --daemonize run the application in the background
-u USER, --user USER change to USER[:GROUP] after daemonizing
-p PIDFILE, --pidfile PIDFILE
write PID to PIDFILE after daemonizing
$ python daemon.py -d -vv
About to daemonize
While the cli modules provide a simple API for designing your own applications, the default implementations are intended to be flexible enough to cover most use cases. No matter which cli.app.Application you use, the basic pattern is the same: create a callable that does the work, wrap it in an cli.app.Application, add some parameters and call its run() method.
Your callable may be a simple function or a more complex class that implements the __call__() protocol. Either way, it should accept a single app instance as its only argument. It will use this object to interact with the application framework, find out what arguments were passed on the command line, log messages, etc.
You can wrap the callable in one of two ways. First, cli.app.Application can be thought of as a decorator (see PEP 318 for more information). For example:
@cli.Application
def yourapp(app):
do_stuff()
If you need to pass keyword arguments to the application, you can still use the decorator pattern:
@cli.CommandLineApp(argv=["-v"])
def yourapp(app):
do_stuff()
If you don’t like decorators (or your interpreter doesn’t support them), you can also simply pass your application callable to the cli.app.Application:
def yourapp(app):
do_stuff()
yourapp = cli.CommandLineApp(yourapp)
Most of the supplied cli.app.Application implementations support parameters. Parameters determine how your users interact with your program on the command line. To add parameters to your application, call add_param() after you’ve wrapped your callable:
yourapp.add_param("-v", "--verbose", help="increase the verbosity", default=0, action="count")
The interface here is the same as that implemented by argparse.ArgumentParser. In this case, an verbose attribute will be created on the app.params object with an integer representing the desired verbosity level.
Once you’ve added all the parameters you need (if any – the default implementations include sensible defaults), simply call the run() method on the wrapped callable. It’s best to do this only if your script is actually being run, so shield it with a conditional:
if __name__ == "__main__":
yourapp.run()
This will allow you to import your application and tweak it programmatically from another script without actually invoking it.
The cli.app module establishes the basis for all of the other applications and is a good place to start when looking to extend Application functionality or to understand the basic API.
An application.
Application constructors should always be called with keyword arguments, though the main argument may be passed positionally (as when Application or its subclasses are instantiated as decorators). Arguments are:
main is the callable object that performs the main work of the application. The callable must accept an Application instance as its sole argument. If main is None, it is assumed that a valid callable will be passed to the __call__() method (when using an Application instance as a decorator). If main is not None, the setup() method will be called, allowing subclasses to customize the order in which certain setup steps are executed.
name is the name of the application itself. If name is not None, the name property will inspect the main callable and use its function or class name.
exit_after_main determines whether the application will call sys.exit() after main completes.
stdin, stderr and stdout are file objects that represent the usual application input and outputs. If they are None, they will be replaced with sys.stdin, sys.stderr and sys.stdout, respectively.
version is a string representing the application’s version.
description is a string describing the application. If description is None, the description property will use the main callable’s __doc__ attribute instead.
argv is a list of strings representing the options passed on the command line. If argv is None, sys.argv will be used instead.
profiler is a Profiler instance. If not None, the profiler will be available to the running application.
In all but a very few cases, subclasses that override the constructor should call Application.__init__() at the end of the overridden method to ensure that the setup() method is called.
A string describing the application.
Unless specified when the Application was created, this property will examine the main callable and use its docstring (__doc__ attribute).
A string identifying the application.
Unless specified when the Application was created, this property will examine the main callable and use its name (__name__ or func_name for classes or functions, respectively).
Clean up after the application.
After main has been called, run() passes the return value to this method. By default, post_run() decides whether to call sys.exit() (based on the value of the exit_after_main attribute) or pass the value back to run(). Subclasses should probably preserve this behavior.
Perform any last-minute configuration.
The pre_run() method is called by the run() method before main is executed. This is a good time to do things like read a configuration file or parse command line arguments. The base implementation does nothing.
Run the application, returning its return value.
This method first calls pre_run() and then calls main, passing it an instance of the Application itself as its only argument. The return value is then passed to post_run() which may modify it (or terminate the application entirely).
Configure the Application.
This method is provided so that subclasses can easily customize the configuration process without having to reimplement the base constructor. setup() is called once, either by the base constructor or __call__().
A command line application.
Command line applications extend the basic Application framework to support command line parsing using the argparse module. As with Application itself, main should be a callable. Other arguments are:
usage is a string describing command line usage of the application. If it is not supplied, argparse will automatically generate a usage statement based on the application’s parameters.
epilog is text appended to the argument descriptions.
The rest of the arguments are passed to the Application constructor.
Add a parameter.
add_param() wraps argparse.ArgumentParser.add_argument(), storing the parameter options in a dictionary. This information can be used later by other subclasses when deciding whether to override parameters.
Parse command line.
During pre_run(), CommandLineApp passes the application’s argv attribute to argparse.ArgumentParser.parse_args(). The results are stored in params.
Configure the CommandLineApp.
During setup, the application instantiates the argparse.ArgumentParser and adds a version parameter (-V, to avoid clashing with -v verbose).
Logging applications use the standard library logging module to handle log messages.
A command-line application that knows how to log.
The LoggingApp further extends the cli.app.CommandLineApp, allowing command line configuration of the application logger. In addition to those supported by the standard cli.app.Application and cli.app.CommandLineApp, arguments are:
stream is an open file object to which the log messages will be written. By default, this is standard output (not standard error, as might be expected).
logfile is the name of a file which will be opened by the logging.FileHandler.
message_format and date_format are passed directly to the CommandLineLogger and are interpreted as in the logging package.
Set the verbosity level and configure the logger.
The application passes the params object to the CommandLineLogger‘s special setLevel() method to set the logger’s verbosity and then initializes the logging handlers. If the logfile attribute is not None, it is passed to a logging.FileHandler instance and that is added to the handler list. Otherwise, if the stream attribute is not None, it is passed to a logging.StreamHandler instance and that becomes the main handler.
Configure the LoggingApp.
This method adds the -l, q, -s and -v parameters to the application and instantiates the log attribute.
Provide extra configuration smarts for loggers.
In addition to the powers of a regular logger, a CommandLineLogger can set its verbosity levels based on a populated argparse.Namespace.
An integer representing the default logging level.
Default: logging.WARN (only warning messages will be shown).
Set the logger verbosity level.
ns is an object with verbose, quiet and silent attributes. verbose and quiet may be positive integers or zero; silent is True or False. If silent is True, the logger’s level will be set to silent_level. Otherwise, the difference between quiet and verbose will be multiplied by 10 so it fits on the standard logging scale and then added to default_level.
An integer representing the silent logging level.
Default: logging.CRITICAL (only critical messages will be shown).
Daemonizing applications run in the background, forking themselves off.
A command-line application that knows how to daemonize.
The DaemonizingApp extends the LoggingApp (for it’s not very helpful to daemonize without being able to log messages somewhere). In addition to those supported by the standard Application, CommandLineApp and LoggingApp, arguments are:
pidfile is a string pointing to a file where the application will write its process ID after it daemonizes. If it is None, no such file will be created.
chdir is a string pointing to a directory to which the application will change after it daemonizes.
null is a string representing a file that will be opened to replace stdin, stdout and stderr when the application daemonizes. By default, this os.path.devnull.
Run in the background.
daemonize() must be called explicitly by the application when it’s ready to fork into the background. It forks, flushes and replaces stdin, stderr and stdout with the open null file and, if requested on the command line, writes its PID to a file and changes user/group.
Configure the DaemonizingApp.
This method adds the -d, u, and -p parameters to the application.
cli ships with a number of unit tests that help ensure that the code run correctly. The tests live in the tests package and can be run by setup.py:
$ python setup.py test
You can get a sense for how completely the unit tests exercise cli by running the coverage tool:
$ coverage run --branch setup.py test
coverage tracks the code statements and branches that the test suite touches and can generate a report listing the lines of code that are missed:
$ coverage report -m --omit "tests,/home/will/lib,lib/cli/ext,setup"
It’s useful to omit the third party code directory (ext) as well as the path to the Python standard library as well as setup.py and the tests themselves.
All new code in cli should be accompanied by unit tests. Eventually, the unit tests should be complemented by a set of functional tests (especially to stress things like the daemon code).