Build File¶

The build file is a YAML file that describes the build process.

There are two types of build files:

Continuous Integration (CI) Build File
Deploy Build File

Continuous Integration (CI) Build File¶

The CI build file is a YAML file that describes the build process for a continuous integration (CI) system.

The CI build file is divided into the following sections:

py_ci: The version of the build file. This is a required field.
type: The type of the build file. This is a required field.
options: The options for the build file.
- venv: The virtual environment options.
- interpreters: The interpreters to use.
stages: The stages of the build process.

Type¶

The type of the build file is always ci and the version is the version of py-ci, currently only version 1.0 is available.

ci_build_file.yaml¶

type: ci
version: "1.0"

Options¶

The options for the build file are as follows:

interpreters: The interpreters to use. This is a list of strings for the interpreter versions. At least one is required.
ci_build_file.yaml continued¶
```
interpreters:
  - python3.11
```
venv: The virtual environment options.
- path: The path to the virtual environment. This will become the {venv_path} variable and can use the {interpreter} variable.
- scripts: The path to the scripts directory in the virtual environment. This will become the {venv_scripts} variable and can use the {venv_path} and {interpreter} variables.
ci_build_file.yaml continued¶
```
options:
  venv:
    path: ./venv/{interpreter}
    scripts: ./{venv_path}/bin
```

Stages¶

The stages of the build process are arbitrary and can be named anything. Each stage consists of a list of commands to execute on the build system shell.

The environment variables on the user system are available to the build file, as well as the following variables for each interpreter specified earlier:

{venv_scripts}: The path to the scripts directory in the virtual environment.
{venv_path}: The path to the virtual environment.
{interpreter}: The interpreter version. (e.g., "python3.11")

ci_build_file.yaml continued¶

stages:
  before_install:
    - ./{venv_scripts}/python -m pip install -U pip wheel setuptools
    - ./{venv_scripts}/pip install poetry
  install:
    - ./{venv_scripts}/poetry install
  quality_checks:
    - ./{venv_scripts}/poetry run pytest
  publish:
    - mkdir -p ./build/{interpreter}
    - cp ./coverage.xml ./build/{interpreter}/coverage.xml
    - cp ./junit.xml ./build/{interpreter}/junit.xml
    - cp ./.coverage ./build/{interpreter}/.coverage
    - cp -R ./htmlcov ./build/{interpreter}/htmlcov
  cleanup:
    - rm -R ./{venv_path}

Full Example¶

ci_build_file.yaml¶

py_ci: "1.0"
type: ci
options:
  venv:
    path: ./venv/{interpreter}
    scripts: ./{venv_path}/bin
interpreters:
  - python3.11
stages:
  before_install:
    - ./{venv_scripts}/python -m pip install -U pip wheel setuptools
    - ./{venv_scripts}/pip install poetry
  install:
    - ./{venv_scripts}/poetry install
  quality_checks:
    - ./{venv_scripts}/poetry run pytest
  cleanup:
    - rm -R ./{venv_path}

Deploy Build File¶

The deploy build file is a YAML file that describes the build process for deploying an application in a continuous development (CD) system.

The deploy build file is divided into the following sections:

py_ci: The version of the build file. This is a required field.
type: The type of the build file. This is a required field.
connections: The connections to use.
variables: The variables to use.
stages: The stages of the build process.

Type¶

The type of the build file is always deploy and the version is the version of py-ci, currently only version 1.0 is available.

deploy_build_file.yaml¶

type: deploy
version: "1.0"

Connections¶

The connection section is a list of connections to deploy to. Each connection has the following options:

type: The type of connection to use. Currently, only ssh is supported.
host: The host to connect to, in the format of user@host[:port]. This can use environment variables.
client: The client options.
- key_type: The type of key to use. RSA, DSSKey, ECDSAKey, and Ed25519 are supported.
- key: The key to use. This can use environment variables. (Must provide this or the keyfile option.)
- keyfile: The key file to use. This can use environment variables. (Must provide this or the key option.)
- passphrase: The passphrase to use. This can use environment variables. This can be omitted if the key does not have a passphrase.

deploy_build_file.yaml continued¶

connections:
  - type: ssh
    host: "{DEPLOY_HOST}"
    client:
      key_type: RSA
      # In this example, the RSA key is stored in the RSA_KEY environment variable.
      key: "{RSA_KEY}"
      passphrase: "{RSA_PASSPHRASE}" # Can be omitted if the key does not have a passphrase.

Variables¶

The variables section is to define variables to use in the build file. These variables can be used in the stages section. This section can use environment variables, as well as other variables defined before it from within the variables section.

deploy_build_file.yaml continued¶

variables:
  host_venv: "{DEPLOY_TO}/venv" # Here, DEPLOY_TO is an environment variable.
  deploy_bin: "{host_venv}/bin" # We can use the host_venv variable here.
  dist_dir: ./dist
  wheel_file: "*.whl"

Stages¶

As with the CI build file, the stages of the build process are arbitrary and can be named anything. Each stage consists of a list of commands to execute, either locally (the default) or remotely.

The environment variables on the user system are available to the build file, as well as the variables defined in the variables section.

deploy_build_file.yaml continued¶

stages:
  clean:
    - rm -R {dist_dir}/* # Local command
  build:
    - poetry build # Local command

Remote Commands¶

To run commands on the remote host, use the !target remote directive. To change back to running commands locally, use the !target local directive.

deploy_build_file.yaml stages section, continued¶

  prepare_host:
    # Set the run target to remote
    - !target remote
    - python3 -m virtualenv "{host_venv}" # The host_venv variable from the variables section is used here.
    # Change it back to local
    - !target local
    - echo "Host prepared"

This could also be done using the !local and !remote directives, regardless of any previous !target directive.

Alternative deploy_build_file.yaml stages section, continued, using the !local and !remote directives¶

  prepare_host:
    - !remote python3 -m virtualenv "{host_venv}"
    - !local echo "Host prepared"

Note

Variables accessible using the {} syntax are available in the remote commands, including environment variables from the local system, but not including environment variables from the remote system.

Transfer Files¶

To transfer files to the remote host, use the !put directive. This directive has the following arguments:

source: The source file or directory to transfer. This supports a glob pattern to put multiple files.

Warning

The glob pattern must start with a * or ** to work correctly. This will be fixed in a future release.
dest: The destination directory on the remote host.

All arguments can use variables from the variables section and environment variables from the local system.

deploy_build_file.yaml stages section, continued¶

  put_to_host:
    - !put
      source: "{dist_dir}/{wheel_file}"
      dest: "{DEPLOY_TO}"

Full Example¶

deploy_build_file.yaml¶

# Expects DEPLOY_HOST, RSA_KEY, RSA_PASSPHRASE, and DEPLOY_TO environment variables
# to be set in the CI/CD system.
py_ci: "1.0"
type: deploy
connections:
  - type: ssh
    host: "{DEPLOY_HOST}"
    client:
      key_type: RSA
      # In this example, the RSA key is stored in the RSA_KEY environment variable.
      key: "{RSA_KEY}"
      passphrase: "{RSA_PASSPHRASE}"
variables:
  host_venv: "{DEPLOY_TO}/venv" # Here, DEPLOY_TO is an environment variable.
  deploy_bin: "{host_venv}/bin" # We can use the host_venv variable here.
  dist_dir: ./dist
  wheel_file: "*.whl"
stages:
  clean:
    - rm -R {dist_dir}/* # Local command
  build:
    - poetry build # Local command
  prepare_host:
    # Set the run target to remote
    - !target remote
    - python3 -m virtualenv "{host_venv}"
    # Change it back to local
    - !target local
    - echo "Host prepared"
# prepare_host:
#   - !remote python3 -m virtualenv "{host_venv}"
#   - !local echo "Host prepared"
  put_to_host:
    - !put
      source: "{dist_dir}/{wheel_file}"
      dest: "{DEPLOY_TO}"
  install:
    - !target remote
    - "{deploy_bin}/pip install --upgrade --force-reinstall $(ls -1v {DEPLOY_TO}/{wheel_file} | tail -n 1)"
  clean_host:
    - !target remote
    - rm -R {DEPLOY_TO}/{wheel_file}

PyCI

Navigation

Related Topics

Build File¶

Continuous Integration (CI) Build File¶

Type¶

Options¶

Stages¶

Full Example¶

Deploy Build File¶

Type¶

Connections¶

Variables¶

Stages¶

Remote Commands¶

Transfer Files¶

Full Example¶