PyBuilder Usage Documentation

Introduction

PyBuilder is a multi-purpose software build tool. Most commonly it targets the building and management of software with a strong focus on Python.

Building Python Projects

Some of the capabilities provided by PyBuilder out-of-the box are:

Automatic execution of unit and integration tests on every build
Automatic analysis of the code coverage
Automatic execution and result interpretation of analysis tools, such as flake8
Automatic generation of distutils script setup.py

The general idea is that everything you do in your continuous integration chain, you also do locally before checking in your work.

Why Another Build Tool

When working on large scale software projects based on Java and Groovy I delved into the build process using tools such as Apache Ant, Apache Maven or Gradle. Although none of these tools is perfect they all provide a powerful and extensible way for building and testing software.

When focusing on Python I looked for a similar tool and got frustrated by the large number of tools that all match some aspect of the build and test process. Unfortunately, many of those tools were not suitable for composition and there was no central point of entry.

I suddenly found myself writing “build scripts” in Python over and over again using the tools I found out to be useful.

PyBuilder was born on the attempt to create a reusable tool that should

Make simple things simple
Make hard things as simple as possible
Let me use whatever tool I want to integrate
Integrate these tools into a common view
Let me use Python (which is really great) to write my build files

Concepts

PyBuilder executes build logic that is organized into tasks and actions.

Tasks are the main building blocks of the build logic. A task is an enclosed piece of build logic to be executed as a single unit. Each task can name a set of other tasks that it depends on. PyBuilder ensures that a task gets executed only after all of its dependencies have been executed.

Actions are smaller pieces of build logic than tasks. They are bound to the execution of task. Each action states that it needs to be executed before or after a named task. PyBuilder will execute the action if and only if the named task is executed, either directly or through another tasks’ dependencies.

Actions as well as tasks are decorated plain Python functions. Thus, you can structure your code the way you like if you provide a single point of entry to a build step.

Both task and action functions can request parameters known to PyBuilder through dependency injection by parameter name.

Writing Tasks

Writing a task is easy. You just create a function and decorate with the @task decorator, and add it to your build.py:

from pybuilder.core import task

@task
def say_hello ():
    print "Hello, PyBuilder"

There is now a new task named say_hello available to you. You can verify this by running pyb -t.

Dependency Injection

PyBuilder supports dependency injection for tasks based on parameter names. The following parameters can be used to receive components:

logger: A logger instance which can be used to issue messages to the user.
project: An instance of the project that is currently being built.

Thus we can rewrite the task above to use the logger:

from pythonbuilder.core import task

@task
def say_hello (logger):
   logger.info("Hello, PyBuilder")

Project-specific configuration

Initializers

The configuration of a project is done by mutating the project object. You can access this object from within build.py by writing an initializer. An initializer is a plain python function that is decorated to become an initializer:


from pybuilder.core import init
@init
def initialize(project):
    pass

Pybuilder always collects and calls initilizers from build.py sorted by alphabetical order. This fact could be used for initilizers managing.


from pybuilder.core import init

@init
def initialize2(project):
    pass
    
@init
def initialize1(project):
    pass

[DEBUG] Registering initializer 'initialize1'
[DEBUG] Registering initializer 'initialize2'
....
[DEBUG] Executing initializer 'initialize1' from 'build'
[DEBUG] Executing initializer 'initialize2' from 'build'

pyb command apply project option -E <environment>, --environment=<environment> which could be used to define environment specific initializers.


from pybuilder.core import init
@init(environments="dev")
def initialize_dev_env(project):
    pass

So initializer initialize_dev_env will be called only if pyb is called with project option --environment=dev.

Project option -E <environment>, --environment=<environment> can be used multiple time.

Project Attributes

Project attributes are values that describe a project. Unlike the properties below, they are not used to configure plugins but rather to describe the project. Each project has several default attributes like version and license. These can be set in the build.py:


name = "myproject"
version = "0.1.14"

Or from within an initializer:

@init
def initialize(project):
    project.version = "0.1.14"

A project’s attributes affect the build in a variety of ways. For instance the license attribute is used when generating a setuptools script to correctly fill the metadata fields. A notable use case for project attributes is replacing placeholder values in source files at build-time with the filter_resources plugin.

Project Version Attribute

The version has to be specified by PEP-440. Additionally, PyBuilder provides Apache Maven logic for versions with suffix .dev by adding generated timestamp label after that which guarantees unique increasing version of distribution. For example, project with version 0.1.14.dev will be built with version 0.1.14.dev20171004032551.

Project Properties

A property is identified by a key (the name of the property, which is a string) and has a value. The type of a property value can be any valid python type.

Project properties are used to configure plugins. Plugins that rely on properties usually set a default value, that you can override. This is conform to the idea of convention over configuration.

For instance the unittest plugin ships with a default property unittest_module_glob set to "*_tests". If the default value does not suit you you can override it by setting the property to something else.

This is done by using the set_property method of the project object. You should do this from within an initializer like so:

@init
def initialize(project):
     project.set_property('unittest_module_glob', '*_unittest')

A complete reference of the available properties is included in the plugin reference

Setting Properties from tasks

Tasks should always bring a sane default for mandatory properties. Setting properties is done from an initializer, just like in build.py. Note that setting project properties from within a task function is possible but will override user-specified properties because initializers run before tasks are executed. Thus, as a general rule, functions decorated with task should only read project properties using project.get_property.

Setting Properties from the command line

Properties can be set or overridden using command line switches.

$ pyb -P spam="spam message"

This command sets/ overrides the property with the name spam with the value spam message.

Note that command line switches only allow properties to be set/ overridden using string values.