Using OS Package Managers
- osdeps files
- The Platform's Package Manager
- OS Independent Package Managers
- The RubyGem Package Manager
- The PIP Package Manager
- Interactions Between Source and OSDep Packages
- Reusing other osdeps entries
- osrepos files
While the main purpose of Autoproj is to build packages from source, there is an obvious need to be able to rely on prebuilt packages offered by the underlying operating system and by other package managers. Autoproj integrates well with those. One may also automatically fall back to source-based packages on platforms where the prebuilt package is either not suitable (e.g. too old or too new), or not available.
The main purpose of the osdeps system is to provide a mapping from a package name (which is arbitrary, specific to the Autoproj workspace) to the name of the package(s) that should be installed using the external package manager. We will see examples of this in follow-up sections.
Packages defined within the osdep system can be used as dependencies for source
packages, by adding them with
<depend name="" /> in the package's
There are mainly two types of "os dependencies" (a.k.a. osdeps) in autoproj. First is the package system offered by the platform itself (e.g. APT on Ubuntu). Second is external package managers that can be installed on top of a platform, such as RubyGems for Ruby.
OS dependencies ("osdeps") definition files are YAML files saved either in the
package sets – usually present in the
autoproj/remotes/ folder of your workspace – or in the
main build configuration, that is in the
itself. We recommend to create an empty
packages.osdeps file in all new
package sets for this purpose.
The general format of an osdep definition file is that of:
package_name: … osdep definition …
Unlike with the version control
entries, osdep files both define
which packages are available and what is the definition of these packages.
Like the version control entries, Autoproj resolves the package in all
osdep files it can find (first in package sets as ordered by the
autoproj/manifest and then in the build configuration). Entries
override each other, and the last entry wins.
One can always inspect the state of an osdep package with
$ autoproj show thor the osdep 'thor' apt-dpkg: ruby-thor 1 matching entries: in /home/doudou/dev/vanilla/rock-website/autoproj/remotes/rock.core/rock.osdeps: debian: ruby-thor ubuntu: precise: gem default: ruby-thor default: gem
This section will be all about the OS dependencies: first how to declare them and then general "cookbook-style" patterns that offer fine-grained setup of the osdeps usage.
The Platform's Package Manager
When using the platform package manager, one does not list the package manager's name, but resolves packages by the platform name and (optionally) version. Autoproj auto-detects the platform, and knows which package manager should be used. For instance, Ubuntu 16.04 would be listed as:
package_name: ubuntu: '16.04': # The quotes are needed, or YAML interprets 16.04 as a number. - name_of_apt_package - name_of_another_apt_package
Autoproj supports the following platforms. The first word is the codename for the platform, that should be used in the osdeps files. The rest gives details about which package manager is being used.
Respectively Ubuntu and Debian, using APT. In addition to the package name, a package version constraint can appended to a package name, e.g.
Gentoo using emerge
MacOSX HomeBrew, used by default when running on MacOSX.
MacPorts can be used instead of HomeBrew on MacOSX by setting
Arch, using pacman
RedHat Entreprise Linux and Fedora, using
OpenSUSE using zipper
FreeBSD using pkg
How Autoproj detects your system, and which package manager it will attempt to use can be found with
$ autoproj osdeps --system-info OS Names: ubuntu, debian OS Versions: 16.04, 16.04.2, lts, xenial, xerus OS Package Manager: apt-dpkg Available Package Managers: gem, pip
Because some platforms share an underlying system (and a lot of packages) with
others, there are also some fallbacks. In the example above, the Ubuntu system
will use the
debian entries if there are no
ubuntu entries. Additionally,
most Debian derivatives will attempt to install the
debian packages even if
they have no specific detection logic for them. A RHEL system will use a
fedora entry if no
rhel entry is present
default entry allows to match "any version not explicitely
listed". For instance, the following
default entry would match any Ubuntu
version that is not 15.10:
package_name: ubuntu: '15.10': - name_of_apt_package - name_of_another_apt_package default: - name_of_apt_package
Multiple platform names and versions can be listed in the keys, separated by commas. The following will in effect provide both a "neither 15.04 nor 15.10" entry for Ubuntu and share that entry with all Debian-based, non-Ubuntu, systems.
package_name: ubuntu,debian: '15.04,15.10': - name_of_apt_package - name_of_another_apt_package default: - name_of_apt_package
OS Independent Package Managers
Autoproj supports a few package managers that are not tied to a specific platform. These package managers have codenames that are used within the osdep files to refer to them:
gem: RubyGems (Ruby packages)
pip: PIP (Python packages)
The most common usage for them is to simply install a package:
package_name: manager_name: - manager_package_name - another_manager_package_name
However, one sometimes wants to also scope the os-independent packages by the platform or platform+version on which autoproj runs. This is supported simply by putting the os-independent package definition in the relevant part of the tree:
package_name: ubuntu: '16.04: - gem: a_gem_package # Installed only on Ubuntu 16.04 gem: another_gem_package # Installed on all Ubuntus gem: yagp # Installed on all operating systems
The RubyGem Package Manager
This package manager allows to install RubyGems, the Ruby programming language's mainstream package manager. Autoproj delegates the management of these packages to Bundler.
The installed gems are shared across all your workspaces, in the
$HOME/.autoproj/gems folder. Bundler ensures that the right versions are
enabled for a given workspace, regardless of gems that might have been
installed by other workspaces. This speeds up bootstrapping quite a bit, as the
common gem install location ensures that they have to be installed only once.
The Gemfile and the bundler binstubs are installed in the workspace's
automatically added to the PATH.
If you simply want to install the latest version of a gem, add it under the
package_name: gem: "package_name"
The package manager supports passing the same arguments that are available to
gem statement in Bundler's Gemfile. To use those, pass a map to the
name key for the gem name and, optionally, the
version key for
a RubyGems version constraint.
package_name: gem: - name: "package_name" version: "~>2.0"
For instance, to use the gem version from git when under ubuntu 20.04 or later, and a particular version of the gem under ubuntu 18.04:
package_name: ubuntu: "18.04": gem: - name: "package_name" version: "~> 2.0" "default": gem: - name: "package_name" git: "https://github.com/someorg/package_name"
Because Bundler sets up the environment so that it is completely isolated from
the other workspace's, gems that are not part of the osdeps system - installed
for instance for debugging purposes – need to be added in a separate
You must run
autoproj osdeps after changing this file.
Unlike the osdeps files,
autoproj/Gemfile follows Bundler's Gemfile
syntax. You do not need to add the
"https://rubygems.org" command, which is already set by Autoproj.
autoproj/Gemfile is usually not checked in the version control system - if
you need a permanent dependency installed, define an osdep for it and depend on
The PIP Package Manager
This package manager allows to install Python packages using pip. The packages
are installed in the workspace's prefix under
automatically added to the PATH.
Interactions Between Source and OSDep Packages
The most common source/osdep package interaction is that any source package can
depend on an osdep package, listed in a
<depend …> tag in the package
Autoproj also allows you to define an osdep package with the same name as a source package. When this happens autoproj will:
- install the osdep package on platforms where it is available
- use the source package otherwise.
If the osdep and source packages are already defined and don't have the same name (for any reason), one can alias the osdep package to match the source package name. If the behavior is undesired (i.e. if one wants to force the installation of the source package), it can also be done (but with caveats ! Read that part well !)
Reusing other osdeps entries
osdep keyword can be used at any place an os-independent package managers
can be used. The package named under this keyword are other osdep entries, that
are recursively resolved by autoproj.
See the cookbook below for examples
osdeps system cookbook
osdep special keyword, one can alias a name to another:
old_name: ubuntu: ... new_name: osdep: old_name
This is commonly used to ensure that an osdep entry matches a source package's name to use the source/osdep fallback mechanism.
Installing platform packages to support an OS-independent package
Some OS-independent package sometime depend on other platform-dependent
software to be installed. The
for instance requires the ruby-dev packages to be installed as well as
This should be done using the
osdep special keyword as
libxml: ubuntu: - libxml2-dev libxslt: ubuntu: - libxslt1-dev nokogiri: gem: nokogiri osdep: - libxml - libxslt
Forcing usage of a source package even if a corresponding osdep package exists
One can use 'nonexistent' instead of a list of packages to force the osdep system
to assume that the package is not available. This can be used to force using a
source package instead of an osdep package. The best place to do so is by
overrides.osdeps file in your main build configuration and add an
entry for the package:
It is important to note at this point that all the package installation is delegated to the underlying package system. When forcing usage of a source package instead of an osdep, Autoproj will not require the installation of said package. However, if other OS packages depend on it, it will still get installed.
Future-proof OS version entries
The best way to format OS version entries in order to have them as future-proof
as possible is to explicitly list old versions and then use the current version
default. This works best because the package names rarely change.
For instance, the gdal package was, starting on Ubuntu 14.04, named
libgdal1-dev. Assuming it was added at the time, its osdep entry would be:
gdal: ubuntu: default: - libgdal1-dev
However, from 16.04 onwards, the package got renamed into
should update the osdep entry to match:
gdal: ubuntu: '14.04,14.10,15.04,15.10': - libgdal1-dev default: - libgdal-dev
This way, from 14.04 up to 15.10, one did not have to change the entry. When 16.04 got out, we needed to update the entry, but won't have to do it until the package name changes again.
Combined with the
nonexistent keyword, this also allows to fall back to the source
package for older OS versions:
control/visp: ubuntu: '14.04,14.10,15.04,15.10': nonexistent default: - libvisp-core3.0-dev
In addition to installing OS packages, Autoproj may also automatically setup os-specific package repositories. In order to do that, you will have to create a YAML file with the .osrepos extension either in a package set or in the main configuration. Similarly to an .osdeps file, the .osrepos should have the format:
- distribution: - version: … repository definition …
Just like with OS packages, Autoproj will automatically detect your operating system and only setup the relevant repositories. Currently, the only supported system is ubuntu/debian.
Apt repository definition
The format of an Apt repository definition is as follows:
- ubuntu: - xenial: - type: repo repo: 'deb http://packages.ros.org/ros/ubuntu xenial main' - type: repo repo: 'deb http://packages.osrfoundation.org/gazebo/ubuntu-stable xenial main' - type: key keyserver: 'hkp://ha.pool.sks-keyservers.net:80' id: 421C365BD9FF1F717815A3895523BAEEB01FA116 - type: key url: 'http://packages.osrfoundation.org/gazebo.key' id: D2486D2DD83DB69272AFE98867170598AF249743
As you can see from the snippet above, Autoproj is capable of adding both the repository
itself and the required keys. If you want to use the same entry for different OS releases you
may use the
default keyword or a comma-separated list of release names/versions:
- ubuntu: - xenial,'14.04',saucy: - type: repo repo: 'deb http://packages.microsoft.com/repos/vscode stable main' - default: - type: repo repo: 'deb http://dl.google.com/linux/chrome/deb/ stable main'
By default, Autoproj will add new entries to
may be changed with the addition of a
file option in the entry. So, to have Autoproj add
new definitions to a
- ubuntu: - xenial: - type: repo repo: 'deb http://packages.ros.org/ros/ubuntu xenial main' file: package_set.list
Existing entries will be left unchanged in the files they are currently defined in and commented out entries will be uncommented in the files they are currently defined in.
Next: now that you have a deeper understanding of what are packages and how they are integrated in an autoproj build let's discuss how to organize the overall build configuration itself