You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@buildstream.apache.org by tv...@apache.org on 2021/02/10 09:00:48 UTC
[buildstream] branch tristan/user-config-docs created (now e1cb389)
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a change to branch tristan/user-config-docs
in repository https://gitbox.apache.org/repos/asf/buildstream.git.
at e1cb389 src/buildstream/data/userconfig.yaml: Remove some comments.
This branch includes the following new commits:
new 4fe3e96 _context.py: Remote `prompt` as a valid configuration key
new 7521348 _frontend/widget.py: Format full element names in `bst show --format %{deps}`
new 527e50f doc/source/using_config.rst: Completely and thoroughly document user config.
new e1cb389 src/buildstream/data/userconfig.yaml: Remove some comments.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[buildstream] 04/04: src/buildstream/data/userconfig.yaml: Remove
some comments.
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch tristan/user-config-docs
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit e1cb389c278d0a8b61802da69d109cfb6a6722a5
Author: Tristan van Berkom <tr...@codethink.co.uk>
AuthorDate: Wed Feb 10 17:59:28 2021 +0900
src/buildstream/data/userconfig.yaml: Remove some comments.
This used to serve as the main documentation for user config, but this
is now thoroughly documented in the docs, we only display this at the
end in order to show what the default configuration is.
---
src/buildstream/data/userconfig.yaml | 55 +++++-------------------------------
1 file changed, 7 insertions(+), 48 deletions(-)
diff --git a/src/buildstream/data/userconfig.yaml b/src/buildstream/data/userconfig.yaml
index d6f91ef..32472ef 100644
--- a/src/buildstream/data/userconfig.yaml
+++ b/src/buildstream/data/userconfig.yaml
@@ -3,12 +3,6 @@
#
# Work Directories
#
-#
-# Note that BuildStream forces the XDG Base Directory names
-# into the environment if they are not already set, and allows
-# expansion of '~' and environment variables when specifying
-# paths.
-#
# Location to store sources
sourcedir: ${XDG_CACHE_HOME}/buildstream/sources
@@ -26,26 +20,14 @@ workspacedir: .
# Cache
#
cache:
- # Size of the artifact cache in bytes - BuildStream will attempt to keep the
- # artifact cache within this size.
- # If the value is suffixed with K, M, G or T, the specified memory size is
- # parsed as Kilobytes, Megabytes, Gigabytes, or Terabytes (with the base
- # 1024), respectively.
- # Alternatively, a percentage value may be specified, which is taken relative
- # to the isize of the file system containing the cache.
+
+ # Use as much space as possible
quota: infinity
# Whether to pull build trees when downloading element artifacts
pull-buildtrees: False
- # Whether to cache build trees on artifact creation:
- #
- # always - Always cache artifact build tree content
- # auto - Only cache build trees when necessary, e.g., for failed builds
- # never - Never cache artifact build tree content. This is not recommended
- # for normal users as this breaks core functionality such as
- # debugging failed builds and may break additional functionality
- # in future versions.
+ # Whether to cache build trees on artifact creation
#
cache-buildtrees: auto
@@ -67,12 +49,8 @@ scheduler:
# Maximum number of retries for network tasks.
network-retries: 2
- # What to do when an element fails, if not running in
- # interactive mode:
- #
- # continue - Continue queueing jobs as much as possible
- # quit - Exit after all ongoing jobs complete
- # terminate - Terminate any ongoing jobs and exit
+ # Control what to do when a task fails, if not running in
+ # interactive mode
#
on-error: quit
@@ -85,16 +63,10 @@ build:
#
# Maximum number of jobs to run per build task.
#
- # The default behavior when this is set to 0, is to use the
- # maximum number of threads available, with a maximum of 8.
- #
max-jobs: 0
#
- # Control which dependencies to build:
- #
- # plan - Only dependencies required for the build plan
- # all - All dependencies
+ # Control which dependencies to build
#
dependencies: plan
@@ -126,24 +98,11 @@ logging:
# Format string for printing the pipeline at startup, this
# also determines the default display format for `bst show`
#
- # See `bst show --help` for details on which symbols can be
- # used in the format string.
element-format: |
%{state: >12} %{full-key} %{name} %{workspace-dirs}
- # Format string for all log messages.
- #
- # Following symbols can be used in the format string:
- #
- # elapsed - Time spent on this operation
- # elapsed-us - Time spent on this operation (with microsecond precision)
- # wallclock - Current wallclock time
- # wallclock-us - Current wallclock time (with microsecond precision)
- # key - The abbreviated cache key (if all sources are consistent)
- # element - Name of the element
- # action - Type of the message
- # message - The main message text
+ # Format string for log messages.
#
message-format: |
[buildstream] 02/04: _frontend/widget.py: Format full element names
in `bst show --format %{deps}`
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch tristan/user-config-docs
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 75213487a1168f7e6c8c625cd3d516132c17771d
Author: Tristan van Berkom <tr...@codethink.co.uk>
AuthorDate: Wed Feb 10 17:28:02 2021 +0900
_frontend/widget.py: Format full element names in `bst show --format %{deps}`
We live in a junction aware world, let's not accidentally print out simple
element names for dependencies which might live across junction boundaries.
---
src/buildstream/_frontend/widget.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/buildstream/_frontend/widget.py b/src/buildstream/_frontend/widget.py
index d4ae6ab..99c9f42 100644
--- a/src/buildstream/_frontend/widget.py
+++ b/src/buildstream/_frontend/widget.py
@@ -418,17 +418,17 @@ class LogLine(Widget):
# Dependencies
if "%{deps" in format_:
- deps = [e.name for e in element._dependencies(_Scope.ALL, recurse=False)]
+ deps = [e._get_full_name() for e in element._dependencies(_Scope.ALL, recurse=False)]
line = p.fmt_subst(line, "deps", yaml.safe_dump(deps, default_style=None).rstrip("\n"))
# Build Dependencies
if "%{build-deps" in format_:
- build_deps = [e.name for e in element._dependencies(_Scope.BUILD, recurse=False)]
+ build_deps = [e._get_full_name() for e in element._dependencies(_Scope.BUILD, recurse=False)]
line = p.fmt_subst(line, "build-deps", yaml.safe_dump(build_deps, default_style=False).rstrip("\n"))
# Runtime Dependencies
if "%{runtime-deps" in format_:
- runtime_deps = [e.name for e in element._dependencies(_Scope.RUN, recurse=False)]
+ runtime_deps = [e._get_full_name() for e in element._dependencies(_Scope.RUN, recurse=False)]
line = p.fmt_subst(
line, "runtime-deps", yaml.safe_dump(runtime_deps, default_style=False).rstrip("\n")
)
[buildstream] 01/04: _context.py: Remote `prompt` as a valid
configuration key
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch tristan/user-config-docs
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 4fe3e96ffd8181e8f2b0b62c43c739744d55385b
Author: Tristan van Berkom <tr...@codethink.co.uk>
AuthorDate: Wed Feb 10 17:26:04 2021 +0900
_context.py: Remote `prompt` as a valid configuration key
This was originally introduced in 27ca65931d3cf6cf897b623f21ba2363f3c54029,
in order to configure whether the user wanted to be prompted for various
commands.
This has long been discarded and only the validity of the unused key remains.
---
src/buildstream/_context.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/buildstream/_context.py b/src/buildstream/_context.py
index 2466650..1dc89c5 100644
--- a/src/buildstream/_context.py
+++ b/src/buildstream/_context.py
@@ -309,7 +309,6 @@ class Context:
"logging",
"projects",
"cache",
- "prompt",
"workspacedir",
"remote-execution",
]
[buildstream] 03/04: doc/source/using_config.rst: Completely and
thoroughly document user config.
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch tristan/user-config-docs
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 527e50f5d7bb9c0b8cc6e1151aca791ae7a71055
Author: Tristan van Berkom <tr...@codethink.co.uk>
AuthorDate: Wed Feb 10 17:58:24 2021 +0900
doc/source/using_config.rst: Completely and thoroughly document user config.
This YAML was historically mostly only documented in the defaults, leaving
much explanation to be desired, now we document everything that can be
controlled by user configuration thoroughly.
---
doc/source/using_config.rst | 630 ++++++++++++++++++++++++++++++++++++++++----
1 file changed, 585 insertions(+), 45 deletions(-)
diff --git a/doc/source/using_config.rst b/doc/source/using_config.rst
index 4f7b98e..8bbfd2e 100644
--- a/doc/source/using_config.rst
+++ b/doc/source/using_config.rst
@@ -5,20 +5,17 @@
User configuration
==================
User configuration and preferences can be specified in a user provided
-configuration file, and usually also on the command line.
+configuration file, and in most cases these preferences can be overridden
+using :ref:`command line options <commands>`.
-Values specified in a user provided configuration file override the
-defaults, while command line options take precedence over any other
-specified configurations.
+Values that are not specified in a user configuration file assume
+their :ref:`default values <config_defaults>`.
Configuration file
------------------
-Users can provide a configuration file to override parameters in
-the default configuration.
-
-Unless a configuration file is explicitly specified on the command line when
-invoking ``bst``, an attempt is made to load user specific configuration from
+Unless a configuration file is explicitly specified on the :ref:`command line <invoking_bst>`
+when invoking ``bst``, an attempt is made to load user specific configuration from
``$XDG_CONFIG_HOME/buildstream.conf``. On most Linux based systems, the location
will be ``~/.config/buildstream.conf``
@@ -34,6 +31,518 @@ will be ``~/.config/buildstream.conf``
any version will fallback to ``$XDG_CONFIG_HOME/buildstream.conf``.
+
+Working directories
+-------------------
+The working directories control where BuildStream will store data. While
+these will have sane defaults, you may want to change these directories
+depending on your partitioning scheme and where you may have extra space.
+
+Environment variables and ``~/`` home directory expressions are supported
+when specifying working directories.
+
+Working directories are configured at the toplevel of your configuration file, like so:
+
+.. code:: yaml
+
+ #
+ # Configure working directories
+ #
+ sourcedir: ~/buildstream/sources
+
+
+Attributes
+~~~~~~~~~~
+
+* ``sourcedir``
+
+ This is the location where BuildStream stores the source code it downloads
+ for builds.
+
+* ``logdir``
+
+ This is the location where BuildStream stores log files of build command
+ output and basically logs pertaining to any activity BuildStream orchestrates.
+
+* ``cachedir``
+
+ This is the location where BuildStream stores the local *CAS* (*Content Addressable Storage*).
+
+ The *CAS* is used to cache anything and everything which BuildStream may
+ reuse at a later time.
+
+ .. attention::
+
+ While it may be beneficial at times to delete the entire cache directory
+ manually in order to free up disk space, one should keep in mind that
+ the ``sourcedir`` and ``logdir`` are configured as subdirectories of
+ this directory when default configuration is used.
+
+ Take care not to accidentally remove all your cached downloaded sources
+ when deleting your cache.
+
+* ``workspacedir``
+
+ A default location for :ref:`opening workspaces <invoking_workspace_open>`.
+
+ .. note::
+
+ By default this is configured to ``.``, which is to say workspaces are
+ created as a subdirectory of the current working directory by default.
+
+ Because of this, the ``workspacedir`` directory is the only directory
+ which is allowed to be declared as a relative path.
+
+
+.. _config_local_cache:
+
+Local cache control
+-------------------
+Beyond deciding what directory you intend to place the cache, there are
+some controls on what is cached locally and how.
+
+These are controlled by the attributes of a ``cache`` dictionary at the
+toplevel of your configuration file, like so:
+
+.. code::
+
+ #
+ # Control the cache
+ #
+ cache:
+
+ # Allow using as much free space as possible
+ quota: infinity
+
+ # Avoid pulling large amounts of data we don't need locally
+ pull-buildtrees: False
+
+ #
+ # Avoid caching build trees if we don't need them
+ cache-buildtrees: auto
+
+
+Attributes
+~~~~~~~~~~
+
+* ``quota``
+
+ This controls how much data you allow BuildStream to cache locally.
+
+ An attempt will be made to error out instead of exceeding the maximum
+ quota which the user has allowed here. Given that it is impossible for
+ BuildStream to know how much data a given build will create, this quota
+ is implemented on a best effort basis.
+
+ The ``quota`` can be specified in multiple ways:
+
+ * The special ``infinity`` value
+
+ This default value states that BuildStream can use as much space as
+ is available on the filesystem where the cache resides.
+
+ * A number in bytes.
+
+ * A human readable number, suffixed in K, M, G or T
+
+ E.g. ``250K`` being 250 kilobytes, ``100M`` being 100 megabytes, etc.
+
+ * A percentage value, e.g. ``80%``
+
+ Percentage values are taken to represent a percentage of the partition
+ size on the filesystem where the cache has been configured.
+
+* ``pull-buildtrees``
+
+ Whether to pull *build trees* when downloading remote artifacts.
+
+ The *build tree* of an artifact is the directory where a build took
+ place, this is useful for :ref:`running a build shell <invoking_shell>`
+ in order to observe how an element was built or to debug how a
+ build failed if the build failed remotely.
+
+ Since build trees are rather expensive, the default is to not pull
+ build trees for every artifact. If you need a build tree that exists
+ remotely, it will be possible to download it as an option at the
+ time you run a command which requires it.
+
+* ``cache-buildtrees``
+
+ Whether to cache build trees when creating artifacts, if build trees
+ are cached locally and the client is configured to push to remote servers,
+ then build trees will be pushed along with any uploaded artifacts.
+
+ This configuration has three possible values:
+
+ * ``never``: Never cache build trees
+ * ``auto``: Only cache the build trees where necessary (e.g. for failed builds)
+ * ``always``: Always cache the build tree.
+
+
+Scheduler controls
+------------------
+Controls related to how the scheduler works are exposed as attributes of the
+toplevel ``scheduler`` dictionary, like so:
+
+.. code:: yaml
+
+ #
+ # Control the scheduler
+ #
+ scheduler:
+
+ # Allow building up to four seperate elements at a time
+ builders: 4
+
+ # Continue building as many elements as possible if anything fails
+ on-error: continue
+
+
+Attributes
+~~~~~~~~~~
+
+* ``fetchers``
+
+ The number of concurrent tasks which download sources or artifacts.
+
+* ``pushers``
+
+ The number of concurrent tasks which upload sources or artifacts.
+
+* ``builders``
+
+ The number of concurrent tasks which build elements.
+
+ .. note::
+
+ This does not control the number of processes in the scope of the
+ build of a single element, but rather the number of elements which
+ may be built in parallel.
+
+* ``network-retries``
+
+ The number of times to retry a task which failed due to network connectivity issues.
+
+* ``on-error``
+
+ What to do when a task fails and BuildStream is running in non-interactive mode. This can
+ be set to the following values:
+
+ * ``continue``: Continue with other tasks, a summary of errors will be printed at the end
+ * ``quit``: Quit after all ongoing tasks have completed
+ * ``terminate``: Abort any ongoing tasks and exit immediately
+
+ .. note::
+
+ If BuildStream is running in interactive mode, then the ongoing build will be suspended
+ and the user will be prompted and asked what to do when a task fails.
+
+ Interactive mode is automatically enabled if BuildStream is connected to a terminal
+ rather than being run automatically, or, it can be specified on the :ref:`command line <invoking_bst>`.
+
+
+Build controls
+--------------
+Some aspects about how elements get built can be controlled by attributes of the ``build``
+dictionary at the toplevel, like so:
+
+.. code:: yaml
+
+ #
+ # Build controls
+ #
+ build:
+
+ #
+ # Allow up to 4 parallel processes to execute within the scope of one build
+ #
+ max-jobs: 4
+
+
+Attributes
+~~~~~~~~~~
+
+* ``max-jobs``
+
+ This is a best effort attempt to instruct build systems on how many parallel
+ processes to use when building an element.
+
+ It is supported by most popular build systems such as ``make``, ``cmake``, ``ninja``,
+ etc, via environment variables such as ``MAXJOBS`` and similar command line options.
+
+ When using the special value ``0``, BuildStream will allocate the number of threads
+ available on the host and limit this with a hard coded value of ``8``, which was
+ found to be an optimial number when building even on hosts with many cores.
+
+* ``dependencies``
+
+ This instructs what dependencies of the target elements should be built, valid
+ values for this attribute are:
+
+ * ``plan``: Only build elements required to generate the expected target artifacts
+ * ``all``: Build elements even if they are build dependencies of artifacts which are already cached
+
+
+Logging controls
+----------------
+Various aspects of how BuildStream presents output and UI can be controlled with
+attributes of the toplevel ``logging`` dictionary, like so:
+
+.. code:: yaml
+
+ #
+ # Control logging output
+ #
+ logging:
+
+ #
+ # Whether to be verbose
+ #
+ verbose: True
+
+
+Attributes
+~~~~~~~~~~
+
+* ``verbose``
+
+ Whether to use verbose logging.
+
+* ``debug``
+
+ Whether to print messages related to debugging BuildStream itself.
+
+* ``key-length``
+
+ When displaying abbreviated cache keys, this controls the number of characters
+ of the cache key which should be printed.
+
+* ``throttle-ui-updates``
+
+ Whether the throttle updates to the status bar in interactive mode. If set to ``True``,
+ then the status bar will be updated once per second.
+
+* ``error-lines``
+
+ The maximum number of lines to print in the main logging output related to an
+ error processing an element, these will be the last lines found in the relevant
+ element's stdout and stderr.
+
+* ``message-lines``
+
+ The maximum number of lines to print in a detailed message sent to the main logging output.
+
+* ``element-format``
+
+ The default format to use when printing out elements in :ref:`bst show <invoking_show>`
+ output, and when printing the pipeline summary at the beginning of sessions.
+
+ The format is specified as a string containing variables which will be expanded
+ in the resulting string, variables must be specified using a leading percent sign
+ and enclosed in curly braces, a colon can be specified in the variable to perform
+ python style string alignments, e.g.:
+
+ .. code:: yaml
+
+ logging:
+
+ #
+ # Set the element format
+ #
+ element-format: |
+
+ %{state: >12} %{full-key} %{name} %{workspace-dirs}
+
+ Variable names which can be used in the element format consist of:
+
+ * ``name``
+
+ The :ref:`element path <format_element_names>`, which is the name of the element including
+ any leading junctions.
+
+ * ``key``
+
+ The abbreviated cache key, the length of which is controlled by the ``key-length`` logging configuration.
+
+ * ``full-key``
+
+ The full cache key.
+
+ * ``state``
+
+ The element state, this will be formatted as one of the following:
+
+ * ``no reference``: If the element still needs to be :ref:`tracked <invoking_source_track>`.
+ * ``junction``: If the element is a junction and as such does not have any relevant state.
+ * ``failed``: If the element has been built and the build has failed.
+ * ``cached``: If the element has been successfully built and is present in the local cache.
+ * ``fetch needed``: If the element cannot be built yet because the sources need to be :ref:`fetched <invoking_source_fetch>`.
+ * ``buildable``: If the element has all of it's sources and build dependency artifacts cached locally.
+ * ``waiting``: If the element has all of it's sources cached but it's build dependencies are not yet locally cached.
+
+ * ``config``
+
+ The :ref:`element configuration <format_config>`, formatted in YAML.
+
+ * ``vars``
+
+ The resolved :ref:`element variables <format_variables>`, formatted as a simple YAML dictionary.
+
+ * ``env``
+
+ The resolved :ref:`environment variables <format_environment>`, formatted as a simple YAML dictionary.
+
+ * ``public``
+
+ The resolved :ref:`public data <format_public>`, formatted in YAML.
+
+ * ``workspaced``
+
+ If the element has an open workspace, this will expand to the string *"(workspaced)"*, otherwise
+ it will expand to an empty string.
+
+ * ``workspace-dirs``
+
+ If the element has an open workspace, this will expand to the workspace directory, prefixed with
+ the text *"Workspace: "*, otherwise it will expand to an empty string.
+
+ * ``deps``
+
+ A list of the :ref:`element paths <format_element_names>` of all dependency elements.
+
+ * ``build-deps``
+
+ A list of the :ref:`element paths <format_element_names>` of all build dependency elements.
+
+ * ``runtime-deps``
+
+ A list of the :ref:`element paths <format_element_names>` of all runtime dependency elements.
+
+* ``message-format``
+
+ The format to use for messages being logged in the aggregated main logging output.
+
+ Similarly to the ``element-format``, The format is specified as a string containing variables which
+ will be expanded in the resulting string, and variables must be specified using a leading percent sign
+ and enclosed in curly braces, e.g.:
+
+ .. code:: yaml
+
+ logging:
+
+ #
+ # Set the message format
+ #
+ message-format: |
+
+ [%{elapsed}][%{key}][%{element}] %{action} %{message}
+
+ Variable names which can be used in the element format consist of:
+
+ * ``elapsed``
+
+ If this message announces the completion of (successful or otherwise) of an activity, then
+ this will expand to a time code showing how much time elapsed for the given activity, in
+ the format: ``HH:MM:SS``, otherwise an empty time code will be displayed in the format:
+ ``--:--:--``.
+
+ * ``elapsed-us``
+
+ Similar to the ``elapsed`` variable, however the timecode will include microsecond precision.
+
+ * ``wallclock``
+
+ This will display a timecode for each message displaying the local wallclock time, in the
+ format ``HH:MM:SS``.
+
+ * ``wallclock-us``
+
+ Similar to the ``wallclock`` variable, however the timecode will include microsecond precision.
+
+ * ``key``
+
+ The abbreviated cache key of the element the message is related to, the length of which is controlled
+ by the ``key-length`` logging configuration.
+
+ If the message in question is not related to any element, then this will expand to whitespace
+ of equal length.
+
+ * ``element``
+
+ This will be formatted to an indicator consisting of the type of activity which is being
+ performed on the element (e.g. *"build"* or *"fetch"* etc), and the :ref:`element path <format_element_names>`
+ of the element this message is related to.
+
+ If the message in question is not related to any element, then a string will be formatted
+ to indicate that this message is related to a core activity instead.
+
+ * ``action``
+
+ A classifier of the type of message this is, the possible values this will expand to are:
+
+ * ``DEBUG``
+
+ This is a message related to debugging BuildStream itself
+
+ * ``STATUS``
+
+ A message showing some detail of what is currently happening, this message will not
+ be displayed unless verbose output is enabled.
+
+ * ``INFO``
+
+ An informative message, this may be printed for instance when discovering a new
+ ref for source code when running :ref:`bst source track <invoking_source_track>`.
+
+ * ``WARN``
+
+ A warning message.
+
+ * ``ERROR``
+
+ An error message.
+
+ * ``BUG``
+
+ A bug happened in BuildStream, this will usually be accompanied by a python stack trace.
+
+ * ``START``
+
+ An activity related to an element started.
+
+ Any ``START`` message will always be accompanied by a later ``SUCCESS``, ``FAILURE``
+ or ``SKIPPED`` message.
+
+ * ``SUCCESS``
+
+ An activity related to an element completed successfully.
+
+ * ``FAILURE``
+
+ An activity related to an element failed.
+
+ * ``SKIPPED``
+
+ After strating this activity, it was discovered that no work was needed and
+ the activity was skipped.
+
+ * ``message``
+
+ The brief message, or the path to the corresponding log file, will be printed here.
+
+ When this is a scheduler related message about the commencement or completion of
+ an element related activity, then the path to the corresponding log for that activity
+ will be printed here.
+
+ If it is a message issued for any other reason, then the message text will be formatted here.
+
+ .. note::
+
+ Messages issued by the core or by plugins are allowed to provide detailed accounts, these
+ are the indented multiline messages which sometimes get displayed in the main aggregated
+ logging output, and will be printed regardless of the logging ``message-format`` value.
+
+
Remote services
---------------
BuildStream can be configured to cooperate with remote caches and
@@ -63,7 +572,9 @@ The ``auth`` configuration block looks like this:
client-cert: client.crt
client-key: client.key
-**Attributes:**
+
+Attributes
+''''''''''
* ``server-cert``
@@ -129,7 +640,9 @@ Cache server configuration is declared in the following way:
client-cert: client.crt
client-key: client.key
-**Attributes:**
+
+Attributes
+''''''''''
* ``override-project-caches``
@@ -402,7 +915,8 @@ using the ``remote-execution`` key, like so:
url: http://cache.flalback.example.com:50052
instance-name: main
-**Attributes:**
+Attributes
+''''''''''
* ``pull-artifact-files``
@@ -479,11 +993,12 @@ which looks like this:
.. _user_config_project_overrides:
-Project specific value
-----------------------
+Project specific values
+-----------------------
The ``projects`` key can be used to specify project specific configurations,
the supported configurations on a project wide basis are listed here.
+
.. _user_config_strict_mode:
Strict build plan
@@ -494,7 +1009,6 @@ by default, but recommended to turn off in developer scenarios where
you might want to build a large system and test it quickly after
modifying some low level component.
-
**Example**
.. code:: yaml
@@ -512,7 +1026,7 @@ modifying some low level component.
.. _config_default_mirror:
-Default Mirror
+Default mirror
~~~~~~~~~~~~~~
When using :ref:`mirrors <project_essentials_mirrors>`, a default mirror can
be defined to be fetched first.
@@ -520,9 +1034,9 @@ The default mirror is defined by its name, e.g.
.. code:: yaml
- projects:
- project-name:
- default-mirror: oz
+ projects:
+ project-name:
+ default-mirror: oz
.. note::
@@ -531,45 +1045,71 @@ The default mirror is defined by its name, e.g.
``--default-mirror`` command-line option.
-.. _config_local_cache:
+Project options
+~~~~~~~~~~~~~~~
+One can specify values to use for :ref:`project options <project_options>` for the projects
+you use here, this avoids needing to specify the options on the command line every time.
-Local cache expiry
-~~~~~~~~~~~~~~~~~~
-BuildStream locally caches artifacts, build trees, log files and sources within a
-cache located at ``~/.cache/buildstream`` (unless a $XDG_CACHE_HOME environment
-variable exists). When building large projects, this cache can get very large,
-thus BuildStream will attempt to clean up the cache automatically by expiring the least
-recently *used* artifacts.
+**Example**
-By default, cache expiry will begin once the file system which contains the cache
-approaches maximum usage. However, it is also possible to impose a quota on the local
-cache in the user configuration. This can be done in two ways:
+.. code:: yaml
-1. By restricting the maximum size of the cache directory itself.
+ projects:
-For example, to ensure that BuildStream's cache does not grow beyond 100 GB,
-simply declare the following in your user configuration (``~/.config/buildstream.conf``):
+ #
+ # Configure the debug flag offered by `project-name`
+ #
+ project-name:
+ options:
+ debug-build: True
-.. code:: yaml
- cache:
- quota: 100G
+Source cache servers
+~~~~~~~~~~~~~~~~~~~~
+As already described in the section concerning configuration of
+:ref:`source cache servers <config_source_caches>`, these can be specified on a per project basis.
+
+
+Artifact cache servers
+~~~~~~~~~~~~~~~~~~~~~~
+As already described in the section concerning configuration of
+:ref:`artifact cache servers <config_artifact_caches>`, these can be specified on a per project basis.
-This quota defines the maximum size of the artifact cache in bytes.
-Other accepted values are: K, M, G or T (or you can simply declare the value in bytes, without the suffix).
-This uses the same format as systemd's
-`resource-control <https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html>`_.
-2. By expiring artifacts once the file system which contains the cache exceeds a specified usage.
+Remote execution configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Following the same format as the toplevel :ref:`remote execution configuration <user_config_remote_execution_service>`,
+the global configuration can be overridden on a per project basis in this project override section.
-To ensure that we start cleaning the cache once we've used 80% of local disk space (on the file system
-which mounts the cache):
+**Example**
.. code:: yaml
- cache:
- quota: 80%
+ projects:
+
+ project-name:
+
+ #
+ # If `project-name` is built as the toplevel project in this BuildStream session,
+ # then use this remote execution configuration instead of any globally defined
+ # remote execution configuration.
+ #
+ remote-execution:
+ pull-artifact-files: True
+ execution-service:
+ url: http://execution.example.com:50051
+ instance-name: main
+
+.. note::
+
+ Only one remote execution service will be considered for any invocation of BuildStream.
+
+ If you are building a project which has a junction into another subproject for which you have
+ specified a project specific remote execution service for in your user configuration, then
+ it will be ignored in the context of building that toplevel project.
+
+.. _config_defaults:
Default configuration
---------------------