Repositories¶
The cppyy
module is a frontend that requires an intermediate (Python
interpreter dependent) layer, and a backend (see
Package Structure).
Because of this layering and because it leverages several existing packages
through reuse, the relevant codes are contained across a number of
repositories.
Frontend, cppyy: https://github.com/wlav/cppyy
CPython (v2/v3) intermediate: https://github.com/wlav/CPyCppyy
PyPy intermediate (module _cppyy): https://foss.heptapod.net/pypy
Backend, cppyy: https://github.com/wlav/cppyy-backend
The backend repo contains both the cppyy-cling (under “cling”) and cppyy-backend (under “clingwrapper”) packages.
Building from source¶
Except for cppyy-cling, the structure in the repositories follows a normal
PyPA package and they are thus ready to build with setuptools: simply
clone the package and either run python setup.py
, or use pip
.
It is highly recommended to follow the dependency chain when manually
upgrading packages individually (i.e. cppyy-cling
, cppyy-backend
,
CPyCppyy
if on CPython, and then finally cppyy
), because upstream
packages expose headers that are used by the ones downstream.
Of course, if only building for a patch/point release, there is no need to
re-install the full chain (or follow the order).
Always run the local updates from the package directories (i.e. where the
setup.py
file is located), as some tools rely on the package structure.
The STDCXX
envar can be used to control the C++ standard version; use
MAKE
to change the make
command; and MAKE_NPROCS
to control the
maximum number of parallel jobs.
Compilation of the backend, which contains a customized version of
Clang/LLVM, can take a long time, so by default the setup script will use all
cores (x2 if hyperthreading is enabled).
On MS Windows, some temporary path names may be too long, causing the build to
fail.
To resolve this issue, point the TMP
and TEMP
envars to an existing
directory with a short name before the build:
For example:
> set TMP=C:\TMP
> set TEMP=C:\TMP
The first package to build is cppyy-cling
.
It may take a long time, especially on a laptop (Mac ARM being a notable
exception), since Cling comes with a builtin version of LLVM/Clang.
Consider therefore for a moment your reasons for building from source: there
being no pre-built wheel for the platform that you’re interested in or simply
needing the latest version from the repository; or perhaps you are planning
to develop/modify the sources.
If the former, clone the repository, check out a specific tagged release as needed, then run the following steps to add Cling and build a wheel. Once built, install the wheel as appropriate:
$ git clone https://github.com/wlav/cppyy-backend.git
$ cd cppyy-backend/cling
$ python setup.py egg_info
$ python create_src_directory.py
$ python setup.py bdist_wheel
$ python -m pip install dist/cppyy_cling-* --upgrade
Note
cppyy-cling
wheels do not depend on the Python interpreter and can
thus be re-used for any version of Python or PyPy.
The egg_info
setup command is needed for create_src_directory.py
to
find the right version.
That script in turn downloads the proper release from upstream, trims and
patches it,
and installs the result in the “src” directory.
When done, the structure of cppyy-cling
looks again like a PyPA package
and can be used/installed as expected, here done with pip
.
By default, the setup script will use all cores (x2 if hyperthreading is
enabled).
You can change this behavior by setting the MAKE_NPROCS
envar to the
desired number of allowable sub jobs.
If on the other hand you are building from source to develop/modify
cppyy-cling
, consider using the cmake
interface.
The first installation will still be just as slow, but subsequent builds can
be incremental and thus much faster.
For this use, first install the latest version from a pre-built wheel, which
will setup the proper directory structure, then use cmake to build and
install the latest or modified version of cppyy-cling
into that:
$ python -m pip install cppyy-cling
$ git clone https://github.com/wlav/cppyy-backend.git
$ cd cppyy-backend/cling
$ python setup.py egg_info
$ python create_src_directory.py
$ mkdir dev
$ cd dev
$ cmake ../src -Wno-dev -DCMAKE_CXX_STANDARD=17 -DLLVM_ENABLE_EH=0 -DLLVM_ENABLE_RTTI=0 -DLLVM_ENABLE_TERMINFO=0 -DLLVM_ENABLE_ASSERTIONS=0 -Dminimal=ON -Druntime_cxxmodules=OFF -Dbuiltin_zlib=ON -Dbuiltin_cling=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=<path to environment python site-packages>
$ make -j <N> install
where the cmake
command needs to be given the full path to
site-packages/cppyy_backend in the virtual environment or other
installation location.
Adjust other options (esp. CMAKE_CXX_STANDARD
) as needed.
For the build command, adjust the cmake
command as appropriate for your
favorite, or platform-specific, build system and/or use cmake --build
instead of make
directly.
See the cmake documentation for details.
Next up is cppyy-backend
(cppyy-backend, subdirectory “clingwrapper”; omit
the first step if you already cloned the repo for cppyy-cling
):
$ git clone https://github.com/wlav/cppyy-backend.git
$ cd cppyy-backend/clingwrapper
$ python -m pip install . --upgrade --no-use-pep517 --no-deps
Note the use of --no-use-pep517
, which prevents pip
from needlessly
going out to pypi.org and creating a local “clean” build environment from the
cached or remote wheels.
Instead, by skipping PEP 517, the local installation will be used.
This is imperative if there was a change in public headers or if the version
of cppyy-cling
was locally updated and is thus not available on PyPI.
Upgrading CPyCppyy
(if on CPython; it’s not needed for PyPy) and cppyy
is very similar:
$ git clone https://github.com/wlav/CPyCppyy.git
$ cd CPyCppyy
$ python -m pip install . --upgrade --no-use-pep517 --no-deps
Just like cppyy-cling
, CPyCppyy
has cmake
scripts which are the
recommended way for development, as incremental builds are faster:
$ mkdir build
$ cmake ../CPyCppyy
$ make -j <N>
then simply point the PYTHONPATH
envar to the build directory above to
pick up the local cppyy.so module.
Finally, the top-level package cppyy
:
$ git clone https://github.com/wlav/cppyy.git
$ cd cppyy
$ python -m pip install . --upgrade --no-deps
Please see the pip documentation for more options, such as developer mode.