:mod:`!compileall` --- Byte-compile Python libraries
====================================================
.. module:: compileall
:synopsis: Tools for byte-compiling all Python source files in a directory tree.
**Source code:** :source:`Lib/compileall.py`
--------------
This module provides some utility functions to support installing Python
libraries. These functions compile Python source files in a directory tree.
This module can be used to create the cached byte-code files at library
installation time, which makes them available for use even by users who don't
have write permission to the library directories.
.. include:: ../includes/wasm-notavail.rst
.. _compileall-cli:
Command-line use
----------------
This module can work as a script (using :program:`python -m compileall`) to
compile Python sources.
.. program:: compileall
.. option:: directory ...
file ...
Positional arguments are files to compile or directories that contain
source files, traversed recursively. If no argument is given, behave as if
the command line was :samp:`-l {<directories from sys.path>}`.
.. option:: -l
Do not recurse into subdirectories, only compile source code files directly
contained in the named or implied directories.
.. option:: -f
Force rebuild even if timestamps are up-to-date.
.. option:: -q
Do not print the list of files compiled. If passed once, error messages will
still be printed. If passed twice (``-qq``), all output is suppressed.
.. option:: -d destdir
Directory prepended to the path to each file being compiled. This will
appear in compilation time tracebacks, and is also compiled in to the
byte-code file, where it will be used in tracebacks and other messages in
cases where the source file does not exist at the time the byte-code file is
executed.
.. option:: -s strip_prefix
.. option:: -p prepend_prefix
Remove (``-s``) or append (``-p``) the given prefix of paths
recorded in the ``.pyc`` files.
Cannot be combined with ``-d``.
.. option:: -x regex
regex is used to search the full path to each file considered for
compilation, and if the regex produces a match, the file is skipped.
.. option:: -i list
Read the file ``list`` and add each line that it contains to the list of
files and directories to compile. If ``list`` is ``-``, read lines from
``stdin``.
.. option:: -b
Write the byte-code files to their legacy locations and names, which may
overwrite byte-code files created by another version of Python. The default
is to write files to their :pep:`3147` locations and names, which allows
byte-code files from multiple versions of Python to coexist.
.. option:: -r
Control the maximum recursion level for subdirectories.
If this is given, then ``-l`` option will not be taken into account.
:program:`python -m compileall <directory> -r 0` is equivalent to
:program:`python -m compileall <directory> -l`.
.. option:: -j N
Use *N* workers to compile the files within the given directory.
If ``0`` is used, then the result of :func:`os.process_cpu_count`
will be used.
.. option:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
Control how the generated byte-code files are invalidated at runtime.
The ``timestamp`` value, means that ``.pyc`` files with the source timestamp
and size embedded will be generated. The ``checked-hash`` and
``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based
pycs embed a hash of the source file contents rather than a timestamp. See
:ref:`pyc-invalidation` for more information on how Python validates
bytecode cache files at runtime.
The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment
variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH``
environment variable is set.
.. option:: -o level
Compile with the given optimization level. May be used multiple times
to compile for multiple levels at a time (for example,
``compileall -o 1 -o 2``).
.. option:: -e dir
Ignore symlinks pointing outside the given directory.
.. option:: --hardlink-dupes
If two ``.pyc`` files with different optimization level have
the same content, use hard links to consolidate duplicate files.
.. versionchanged:: 3.2
Added the ``-i``, ``-b`` and ``-h`` options.
.. versionchanged:: 3.5
Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option
was changed to a multilevel value. ``-b`` will always produce a
byte-code file ending in ``.pyc``, never ``.pyo``.
.. versionchanged:: 3.7
Added the ``--invalidation-mode`` option.
.. versionchanged:: 3.9
Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options.
Raised the default recursion limit from 10 to
:py:func:`sys.getrecursionlimit()`.
Added the possibility to specify the ``-o`` option multiple times.
There is no command-line option to control the optimization level used by the
:func:`compile` function, because the Python interpreter itself already
provides the option: :program:`python -O -m compileall`.
Similarly, the :func:`compile` function respects the :data:`sys.pycache_prefix`
setting. The generated bytecode cache will only be useful if :func:`compile` is
run with the same :data:`sys.pycache_prefix` (if any) that will be used at
runtime.
Public functions
----------------
.. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)
Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
files along the way. Return a true value if all the files compiled successfully,
and a false value otherwise.
The *maxlevels* parameter is used to limit the depth of the recursion; it
defaults to ``sys.getrecursionlimit()``.
If *ddir* is given, it is prepended to the path to each file being compiled
for use in compilation time tracebacks, and is also compiled in to the
byte-code file, where it will be used in tracebacks and other messages in
cases where the source file does not exist at the time the byte-code file is
executed.
If *force* is true, modules are re-compiled even if the timestamps are up to
date.
If *rx* is given, its ``search`` method is called on the complete path to each
file considered for compilation, and if it returns a true value, the file
is skipped. This can be used to exclude files matching a regular expression,
given as a :ref:`re.Pattern <re-objects>` object.
If *quiet* is ``False`` or ``0`` (the default), the filenames and other
information are printed to standard out. Set to ``1``, only errors are
printed. Set to ``2``, all output is suppressed.
If *legacy* is true, byte-code files are written to their legacy locations
and names, which may overwrite byte-code files created by another version of
Python. The default is to write files to their :pep:`3147` locations and
names, which allows byte-code files from multiple versions of Python to
coexist.
*optimize* specifies the optimization level for the compiler. It is passed to
the built-in :func:`compile` function. Accepts also a sequence of optimization
levels which lead to multiple compilations of one :file:`.py` file in one call.
The argument *workers* specifies how many workers are used to
compile files in parallel. The default is to not use multiple workers.
If the platform can't use multiple workers and *workers* argument is given,
then sequential compilation will be used as a fallback. If *workers*
is 0, the number of cores in the system is used. If *workers* is
lower than ``0``, a :exc:`ValueError` will be raised.
*invalidation_mode* should be a member of the
:class:`py_compile.PycInvalidationMode` enum and controls how the generated
pycs are invalidated at runtime.
The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
the ``-s``, ``-p`` and ``-e`` options described above.
They may be specified as ``str`` or :py:class:`os.PathLike`.
If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
level have the same content, use hard links to consolidate duplicate files.
.. versionchanged:: 3.2
Added the *legacy* and *optimize* parameter.
.. versionchanged:: 3.5
Added the *workers* parameter.
.. versionchanged:: 3.5
*quiet* parameter was changed to a multilevel value.
.. versionchanged:: 3.5
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
no matter what the value of *optimize* is.
.. versionchanged:: 3.6
Accepts a :term:`path-like object`.
.. versionchanged:: 3.7
The *invalidation_mode* parameter was added.
.. versionchanged:: 3.7.2
The *invalidation_mode* parameter's default value is updated to ``None``.
.. versionchanged:: 3.8
Setting *workers* to 0 now chooses the optimal number of cores.
.. versionchanged:: 3.9
Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.
Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()``
.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)
Compile the file with path *fullname*. Return a true value if the file
compiled successfully, and a false value otherwise.
If *ddir* is given, it is prepended to the path to the file being compiled
for use in compilation time tracebacks, and is also compiled in to the
byte-code file, where it will be used in tracebacks and other messages in
cases where the source file does not exist at the time the byte-code file is
executed.
If *rx* is given, its ``search`` method is passed the full path name to the
file being compiled, and if it returns a true value, the file is not
compiled and ``True`` is returned. This can be used to exclude files matching
a regular expression, given as a :ref:`re.Pattern <re-objects>` object.
If *quiet* is ``False`` or ``0`` (the default), the filenames and other
information are printed to standard out. Set to ``1``, only errors are
printed. Set to ``2``, all output is suppressed.
If *legacy* is true, byte-code files are written to their legacy locations
and names, which may overwrite byte-code files created by another version of
Python. The default is to write files to their :pep:`3147` locations and
names, which allows byte-code files from multiple versions of Python to
coexist.
*optimize* specifies the optimization level for the compiler. It is passed to
the built-in :func:`compile` function. Accepts also a sequence of optimization
levels which lead to multiple compilations of one :file:`.py` file in one call.
*invalidation_mode* should be a member of the
:class:`py_compile.PycInvalidationMode` enum and controls how the generated
pycs are invalidated at runtime.
The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
the ``-s``, ``-p`` and ``-e`` options described above.
They may be specified as ``str`` or :py:class:`os.PathLike`.
If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
level have the same content, use hard links to consolidate duplicate files.
.. versionadded:: 3.2
.. versionchanged:: 3.5
*quiet* parameter was changed to a multilevel value.
.. versionchanged:: 3.5
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
no matter what the value of *optimize* is.
.. versionchanged:: 3.7
The *invalidation_mode* parameter was added.
.. versionchanged:: 3.7.2
The *invalidation_mode* parameter's default value is updated to ``None``.
.. versionchanged:: 3.9
Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.
.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None)
Byte-compile all the :file:`.py` files found along ``sys.path``. Return a
true value if all the files compiled successfully, and a false value otherwise.
If *skip_curdir* is true (the default), the current directory is not included
in the search. All other parameters are passed to the :func:`compile_dir`
function. Note that unlike the other compile functions, ``maxlevels``
defaults to ``0``.
.. versionchanged:: 3.2
Added the *legacy* and *optimize* parameter.
.. versionchanged:: 3.5
*quiet* parameter was changed to a multilevel value.
.. versionchanged:: 3.5
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
no matter what the value of *optimize* is.
.. versionchanged:: 3.7
The *invalidation_mode* parameter was added.
.. versionchanged:: 3.7.2
The *invalidation_mode* parameter's default value is updated to ``None``.
To force a recompile of all the :file:`.py` files in the :file:`Lib/`
subdirectory and all its subdirectories::
import compileall
compileall.compile_dir('Lib/', force=True)
# Perform same compilation, excluding files in .svn directories.
import re
compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
# pathlib.Path objects can also be used.
import pathlib
compileall.compile_dir(pathlib.Path('Lib/'), force=True)
.. seealso::
Module :mod:`py_compile`
Byte-compile a single source file.
Codebase Browser
cpython