You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@buildstream.apache.org by tv...@apache.org on 2021/01/28 07:43:23 UTC
[buildstream] 01/02: doc: Redocumenting artifact/source cache
servers.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch tristan/change-remote-config
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 76489ef46d0aa4faf7c81b1f9c00054f8275cffe
Author: Tristan van Berkom <tr...@codethink.co.uk>
AuthorDate: Thu Jan 28 13:59:02 2021 +0900
doc: Redocumenting artifact/source cache servers.
---
doc/source/format_project.rst | 161 ++-------
doc/source/using_config.rst | 464 +++++++++++++++++++-------
doc/source/using_configuring_cache_server.rst | 57 +---
src/buildstream/plugins/elements/stack.py | 2 +-
4 files changed, 401 insertions(+), 283 deletions(-)
diff --git a/doc/source/format_project.rst b/doc/source/format_project.rst
index 0216e52..0a7d6ef 100644
--- a/doc/source/format_project.rst
+++ b/doc/source/format_project.rst
@@ -192,165 +192,70 @@ for more detail.
build-gid: 1001
-.. _project_essentials_artifacts:
+.. _project_artifact_cache:
Artifact server
~~~~~~~~~~~~~~~
-If you have setup an :ref:`artifact server <cache_servers>` for your
-project then it is convenient to configure the following in your ``project.conf``
-so that users need not have any additional configuration to communicate
-with an artifact share.
+When maintaining a BuildStream project, it can be convenient to downstream users
+of your project to provide access to a :ref:`cache server <cache_servers>` you maintain.
+
+The project can provide *recommended* artifact cache servers through project configuration
+using the same semantics as one normally uses in :ref:`user configuration <config_cache_servers>`:
.. code:: yaml
#
- # Artifacts
+ # A remote cache from which to download prebuilt artifacts
#
artifacts:
- # A remote cache from which to download prebuilt artifacts
- - url: https://foo.com:11001
- server-cert: server.crt
- # A remote cache from which to upload/download built/prebuilt artifacts
- - url: https://foo.com:11002
- push: true
+ - url: https://foo.com:11001
+ auth:
server-cert: server.crt
- client-cert: client.crt
- client-key: client.key
-
-.. note::
-
- You can also specify a list of different caches here; earlier entries in the
- list will have higher priority than later ones.
-
-The use of ports are required to distinguish between pull only access and
-push/pull access. For information regarding the server/client certificates
-and keys, please see: :ref:`Key pair for the server <server_authentication>`.
-
-.. note::
-
- Buildstream artifact servers have changed since 1.2 to use protocol buffers
- to store artifact information rather than a directory structure, as well as a
- new server API. As a result newer buildstream clients won't work with older
- servers.
+.. attention::
-.. _project_essentials_split_artifacts:
+ Unlike user configuration, the filenames provided in the :ref:`auth <config_remote_auth>`
+ configuration block are relative to the :ref:`project directory <format_structure>`.
-Split cache servers
-~~~~~~~~~~~~~~~~~~~
+ It is recommended to include public keys such as the ``server-cert`` along with your
+ project so that downstream users can have automatic read access to your project.
-Should you need to configure an artifact cache to work with a CAS
-server that does not support BuildStream's artifact format, you can
-"split" that cache and run an artifacts-only server separately. The
-format for that is as such:
-
-.. code:: yaml
-
- #
- # Artifacts
- #
- artifacts:
- # A remote cache from which to download prebuilt artifacts
- - url: https://storage.foo.com:11001
- server-cert: server.crt
- # "storage" remotes store the artifact contents only - this can
- # be a normal CAS implementation such as those provided by
- # Buildbarn, BuildGrid, or Bazel Buildfarm
- type: storage
- - url: https://index.foo.com:11001
- server-cert: server.crt
- # "index" remotes store only artifact metadata. This is
- # currently only provided by the bst-artifact-server
- type: index
- # A remote cache from which to upload/download built/prebuilt artifacts
- - url: https://foo.com:11002
- push: true
- server-cert: server.crt
- client-cert: client.crt
- client-key: client.key
- # Caches that support both can omit the type, or set it to "both" -
- # currently, also only supported by bst-artifact-server
- type: both
+ To provide write access to downstream users, it is recommended that the required
+ private keys such as the ``client-key`` be provided to users out of band,
+ and require that users configure write access separately in their own
+ :ref:`user configuration <config_cache_servers>`.
.. _project_source_cache:
Source cache server
~~~~~~~~~~~~~~~~~~~
-Exactly the same as artifact servers, source cache servers can be specified.
+In the same way as artifact cache servers, the project can provide *recommended* source cache
+servers through project configuration using the same semantics as one normally uses in
+:ref:`user configuration <config_cache_servers>`:
.. code:: yaml
#
- # Source caches
+ # A remote cache from which to download prestaged sources
#
source-caches:
- # A remote cache from which to download prestaged sources
- - url: https://foo.com:11001
- server.cert: server.crt
- # A remote cache from which to upload/download prestaged sources
- - url: https://foo.com:11002
- push: true
+ - url: https://foo.com:11001
+ auth:
server-cert: server.crt
- client-cert: client.crt
- client-key: client.key
-
-.. note::
- Source caches also support "splitting" like :ref:`artifact servers
- <project_essentials_split_artifacts>`.
-
-
-.. _project_remote_execution:
-
-Remote execution
-~~~~~~~~~~~~~~~~
-BuildStream supports remote execution using the Google Remote Execution API
-(REAPI). A description of how remote execution works is beyond the scope
-of this document, but you can specify a remote server complying with the REAPI
-using the `remote-execution` option:
+.. attention::
-.. code:: yaml
+ Unlike user configuration, the filenames provided in the :ref:`auth <config_remote_auth>`
+ configuration block are relative to the :ref:`project directory <format_structure>`.
- remote-execution:
+ It is recommended to include public keys such as the ``server-cert`` along with your
+ project so that downstream users can have automatic read access to your project.
- # A url defining a remote execution server
- execution-service:
- url: http://buildserver.example.com:50051
- instance-name: development-emea-1
- storage-service:
- url: https://foo.com:11002/
- server-cert: server.crt
- client-cert: client.crt
- client-key: client.key
- instance-name: development-emea-1
- action-cache-service:
- url: http://bar.action.com:50052
- instance-name: development-emea-1
-
-storage-service specifies a remote CAS store and the parameters are the
-same as those used to specify an :ref:`artifact server <cache_servers>`.
-
-The action-cache-service specifies where built actions are cached, allowing
-buildstream to check whether an action has already been executed and download it
-if so. This is similar to the artifact cache but REAPI specified, and is
-optional for remote execution to work.
-
-The storage service may be the same endpoint used for artifact
-caching. Remote execution cannot work without push access to the
-storage endpoint though.
-
-Instance name is optional. Instance names separate different shards on
-the same endpoint (url). You can supply a different instance name for
-`execution-service` and `storage-service`, if needed. The instance
-name should be given to you by the service provider of each
-service. Not all remote execution and storage services support
-instance names.
-
-The Remote Execution API can be found via https://github.com/bazelbuild/remote-apis.
-
-Remote execution configuration can be also provided in the `user
-configuration <user_config_remote_execution>`.
+ To provide write access to downstream users, it is recommended that the required
+ private keys such as the ``client-key`` be provided to users out of band,
+ and require that users configure write access separately in their own
+ :ref:`user configuration <config_cache_servers>`.
.. _project_essentials_mirrors:
diff --git a/doc/source/using_config.rst b/doc/source/using_config.rst
index ba38173..c4b4c18 100644
--- a/doc/source/using_config.rst
+++ b/doc/source/using_config.rst
@@ -34,183 +34,421 @@ will be ``~/.config/buildstream.conf``
any version will fallback to ``$XDG_CONFIG_HOME/buildstream.conf``.
-Project specific value
-----------------------
-The ``projects`` key can be used to specify project specific configurations,
-the supported configurations on a project wide basis are listed here.
+Remote services
+---------------
+BuildStream can be configured to cooperate with remote caches and
+execution services.
+
+
+.. _config_remote_auth:
+
+Authentication
+~~~~~~~~~~~~~~
+BuildStream supports end to end encryption when communicating with remote
+services.
+
+All remote service configuration blocks come with an optional ``auth``
+configuration block which allows one to specify the certificates
+and keys required for encrypted traffic.
+
+See the :ref:`server configuration documentation <server_authentication>` for
+details on how the keys can be generated and managed on the server side.
+
+The ``auth`` configuration block looks like this:
+
+.. code:: yaml
+
+ auth:
+ server-cert: server.crt
+ client-cert: client.crt
+ client-key: client.key
+
+**Attributes:**
+
+* ``server-cert``
+
+ The server certificate is used to decrypt traffic coming from the
+ server.
+
+* ``client-cert``
+
+ The client certificate is used by the remote server to decrypt
+ traffic being uploaded to the server.
+
+ The remote server will have it's own copy of this certificate, but the
+ client needs to send this certificate's identity to the server so that
+ the server knows which certificate to use.
+
+* ``client-key``
+
+ The client key is used to encrypt traffic when uploading traffic
+ to the server.
+
+Normally, only the ``server-cert`` is required to securely *download* data
+from remote cache services, while both the ``client-key`` and ``client-cert``
+is required to securely *upload* data to the server.
+
+
+.. _config_cache_servers:
+
+Cache servers
+~~~~~~~~~~~~~
+BuildStream supports two types of cache servers, :ref:`source cache servers <config_source_caches>`
+and :ref:`artifact cache servers <config_artifact_caches>`. These services allow you
+to store sources and build artifacts for later reuse, and share them among your
+peers.
+
+.. important::
+
+ **Storing and indexing**
+
+ Cache servers are split into two separate services, the *index* and the *storage*.
+ Sometimes these services are provided by the same server, and sometimes it is desirable
+ to use different cache servers for indexing and storing data.
+
+ In simple setups, it is possible to use the same cache server for indexing and storing
+ of both sources and artifacts. However, when using :ref:`remote execution <user_config_remote_execution>`
+ it is recommended to use the remote execution build cluster's ``storage-service`` as the *storage*
+ service of your cache servers, which may require setting up your *index* service separately.
+
+ When configuring cache servers, BuildStream will require both storage and indexing capabilities,
+ otherwise no attempt will be made to fetch or push data to and from cache servers.
+
+Cache server configuration is declared in the following way:
+
+.. code:: yaml
+
+ url: https://cache-server.com/cache:11001
+ instance-name: main
+ type: both
+ push: true
+ auth:
+ server-cert: server.crt
+ client-cert: client.crt
+ client-key: client.key
+
+**Attributes:**
+
+* ``url``
+
+ Indicates the ``http`` or ``https`` url and optionally the port number of
+ where the cache server is located.
+
+* ``instance-name``
+
+ Instance names separate different shards on the same endpoint (``url``).
+
+ The instance name is optional, and not all cache server implementations support
+ instance names. The instance name should be given to you by the
+ service provider of each service.
+
+* ``type``
+
+ The type of service you intend to use this cache server for. If unspecified,
+ the default value for this field is ``both``.
+
+ * ``storage``
+
+ Use this cache service for storage.
+
+ * ``index``
+
+ Use this cache service for index content expected to be present in one
+ or more *storage* services.
+
+ * ``both``
+
+ Use this cache service for both indexing and storing data.
+
+
+* ``push``
+
+ Set this to ``true`` if you intend to upload data to this cache server.
+
+ Normally this requires additional credentials in the ``auth`` field.
+
+* ``auth``
+
+ The :ref:`authentication attributes <config_remote_auth>` to connect to
+ this server.
+
+
+.. _config_cache_server_list:
+
+Cache server lists
+''''''''''''''''''
+Cache servers are always specified as *lists* in the configuration, this allows
+*index* and *storage* services to be declared separately, and also allows for
+some redundancy.
+
+**Example:**
+
+.. code:: yaml
+
+ - url: https://cache-server-1.com/index
+ type: index
+ - url: https://cache-server-1.com/storage
+ type: storage
+ - url: https://cache-server-2.com
+ type: both
+
+When downloading data from a cache server, BuildStream will iterate over each
+*index* service one by one until it finds the reference to the data it is looking
+for, and then it will iterate over each *storage* service one by one, downloading
+the referenced data until all data is downloaded.
+
+When uploading data to a cache server, BuildStream will first upload the data to
+each *storage* service which was configured with the ``push`` attribute, and
+upon successful upload, it will proceed to upload the references to the uploaded
+data to each *index* service in the list.
-.. _config_artifacts:
-Artifact server
-~~~~~~~~~~~~~~~
-Although project's often specify a :ref:`remote artifact cache <cache_servers>`
-in their ``project.conf``, you may also want to specify extra caches.
+.. _config_artifact_caches:
-Assuming that your host/server is reachable on the internet as ``artifacts.com``
-(for example), there are two ways to declare remote caches in your user
-configuration:
+Artifact cache servers
+~~~~~~~~~~~~~~~~~~~~~~
+Using artifact :ref:`cache servers <config_cache_servers>` is an essential means of
+*build avoidance*, as it will allow you to avoid building an element which has already
+been built and uploaded to a common artifact server.
-1. Adding global caches:
+Artifact cache servers can be declared in three different ways, with differing
+priorities.
+
+
+Global caches
+'''''''''''''
+To declare the global artifact server list, use the ``artifacts`` key at the
+toplevel of the user configuration.
.. code:: yaml
#
- # Artifacts
+ # Configure a global artifact server for pushing and pulling artifacts
#
artifacts:
- # Add a cache to pull from
- - url: https://artifacts.com/artifacts:11001
- server-cert: server.crt
- # Add a cache to push/pull to/from
- - url: https://artifacts.com/artifacts:11002
+ - url: https://artifacts.com/artifacts:11001
+ push: true
+ auth:
server-cert: server.crt
client-cert: client.crt
client-key: client.key
- push: true
- # Add another cache to pull from
- - url: https://anothercache.com/artifacts:8080
- server-cert: another_server.crt
-.. note::
-
- Caches declared here will be used by **all** BuildStream project's on the user's
- machine and are considered a lower priority than those specified in the project
- configuration.
+Project overrides
+'''''''''''''''''
+To declare artifact servers lists for individual projects, declare them
+in the :ref:`project specific section <user_config_project_overrides>` of
+the user configuration.
-2. Specifying caches for a specific project within the user configuration:
+Artifact server lists declared in this section will only be used for
+elements belonging to the specified project, and will be used instead of
+artifact cache servers declared in the global caches.
.. code:: yaml
+ #
+ # Configure an artifact server for pushing and pulling artifacts from project "foo"
+ #
projects:
- project-name:
+ foo:
artifacts:
- # Add a cache to pull from
- - url: https://artifacts.com/artifacts:11001
- server-cert: server.crt
- # Add a cache to push/pull to/from
- - url: https://artifacts.com/artifacts:11002
+ - url: https://artifacts.com/artifacts:11001
+ push: true
+ auth:
server-cert: server.crt
client-cert: client.crt
client-key: client.key
- push: true
- # Add another cache to pull from
- - url: https://ourprojectcache.com/artifacts:8080
- server-cert: project_server.crt
-.. note::
+Project recommendations
+'''''''''''''''''''''''
+Projects can :ref:`recommend artifact cache servers <project_artifact_cache>` in their
+individual project configuration files.
- Caches listed here will be considered a higher priority than those specified
- by the project. Furthermore, for a given list of URLs, earlier entries will
- have higher priority.
+These will only be used for elements belonging to their respective projects, and
+are the lowest priority configuration.
-Notice that the use of different ports for the same server distinguishes between
-pull only access and push/pull access. For information regarding this and the
-server/client certificates and keys, please see:
-:ref:`Key pair for the server <server_authentication>`.
+.. _config_source_caches:
-.. _config_sources:
+Source cache servers
+~~~~~~~~~~~~~~~~~~~~
+Using source :ref:`cache servers <config_cache_servers>` enables BuildStream to cache
+source code referred to by your project and share those sources with peers who have
+access to the same source cache server.
-Source cache server
-~~~~~~~~~~~~~~~~~~~
-Similarly global and project specific source caches servers can be specified in
-the user configuration.
+This can optimize your build times in the case that it is determined that an element needs
+to be rebuilt because of changes in the dependency graph, as BuildStream will first attempt
+to download the source code from the cache server before attempting to obtain it from an
+external source, which may suffer higher latencies.
+
+Source cache servers can be declared in three different ways, with differing
+priorities.
-1. Global source caches
+
+Global caches
+'''''''''''''
+To declare the global source cache server list, use the ``source-caches`` key at the
+toplevel of the user configuration.
.. code:: yaml
#
- # Source caches
+ # Configure a global source cache server for pushing and pulling sources
#
source-caches:
- # Add a cache to pull from
- - url: https://cache.com/sources:11001
- server-cert: server.crt
- # Add a cache to push/pull to/from
- - url: https://cache.com/sources:11002
+ - url: https://sources.com/sources:11001
+ push: true
+ auth:
server-cert: server.crt
client-cert: client.crt
client-key: client.key
- push: true
- # Add another cache to pull from
- - url: https://anothercache.com/sources:8080
- server-cert: another_server.crt
-2. Project specific source caches
+
+Project overrides
+'''''''''''''''''
+To declare source cache servers lists for individual projects, declare them
+in the :ref:`project specific section <user_config_project_overrides>` of
+the user configuration.
+
+Source cache server lists declared in this section will only be used for
+elements belonging to the specified project, and will be used instead of
+source cache servers declared in the global caches.
.. code:: yaml
+ #
+ # Configure a source cache server for pushing and pulling sources from project "foo"
+ #
projects:
- project-name:
- artifacts:
- # Add a cache to pull from
- - url: https://cache.com/sources:11001
- server-cert: server.crt
- # Add a cache to push/pull to/from
- - url: https://cache.com/sources:11002
+ foo:
+ source-caches:
+ - url: https://sources.com/sources:11001
+ push: true
+ auth:
server-cert: server.crt
client-cert: client.crt
client-key: client.key
- push: true
- # Add another cache to pull from
- - url: https://ourprojectcache.com/sources:8080
- server-cert: project_server.crt
+
+
+Project recommendations
+'''''''''''''''''''''''
+Projects can :ref:`recommend source cache servers <project_source_cache>` in their
+individual project configuration files.
+
+These will only be used for elements belonging to their respective projects, and
+are the lowest priority configuration.
+
.. _user_config_remote_execution:
Remote execution
~~~~~~~~~~~~~~~~
+BuildStream supports building remotely using the
+`Google Remote Execution API (REAPI). <https://github.com/bazelbuild/remote-apis>`_.
+
+You can configure the remote execution services globally in your user configuration
+using the ``remote-execution`` key, like so:
-The configuration for :ref:`remote execution <project_remote_execution>`
-in ``project.conf`` can be provided in the user configuation. The global
-configuration also has a ``pull-artifact-files`` option, which specifies when
-remote execution is being performed whether to pull file blobs of artifacts, or
-just the directory trees required to perform remote builds.
+.. code:: yaml
-There is only one remote execution configuration used per project.
+ remote-execution:
+ pull-artifact-files: True
+ execution-service:
+ url: http://execution.fallback.example.com:50051
+ instance-name: main
+ storage-service:
+ url: https://storage.fallback.example.com:11002
+ instance-name: main
+ auth:
+ server-cert: /keys/server.crt
+ client-cert: /keys/client.crt
+ client-key: /keys/client.key
+ action-cache-service:
+ url: http://cache.flalback.example.com:50052
+ instance-name: main
-The project overrides will be taken in priority. The global
-configuration will be used as fallback.
+**Attributes:**
-1. Global remote execution fallback:
+* ``pull-artifact-files``
-.. code:: yaml
+ This determines whether you want the artifacts which were built remotely
+ to be downloaded into the local CAS, so that it is ready for checkout
+ directly after a built completes.
+
+ If this is set to ``false``, then you will need to download the artifacts
+ you intend to use with :ref:`bst artifact checkout <invoking_artifact_checkout>`
+ after your build completes.
+
+* ``execution-service``
+
+ A :ref:`service configuration <user_config_remote_execution_service>` specifying
+ how to connect with the main *execution service*, this service is the main controlling
+ entity in a remote execution build cluster.
+
+* ``storage-service``
+
+ A :ref:`service configuration <user_config_remote_execution_service>` specifying
+ how to connect with the *Content Addressable Storage* service, this is where build
+ input and output is stored on the remote execution build cluster.
+
+ This service is compatible with the *storage* service offered by
+ :ref:`cache servers <config_cache_servers>`.
+
+* ``action-cache-service``
+
+ A :ref:`service configuration <user_config_remote_execution_service>` specifying
+ how to connect with the *action cache*, this service stores information about
+ activities which clients request be performed by workers on the remote execution
+ build cluster, and results of completed operations.
+
+ This service is optional in a remote execution build cluster, if your remote
+ execution service provides an action cache, then you should configure it here.
+
+
+.. _user_config_remote_execution_service:
- remote-execution:
- execution-service:
- url: http://execution.fallback.example.com:50051
- instance-name: main
- storage-service:
- url: https://storage.fallback.example.com:11002
- server-cert: /keys/server.crt
- client-cert: /keys/client.crt
- client-key: /keys/client.key
- instance-name: main
- action-cache-service:
- url: http://cache.flalback.example.com:50052
- instance-name: main
- pull-artifact-files: True
-
-2. Project override:
+Remote execution service configuration
+''''''''''''''''''''''''''''''''''''''
+Each of the distinct services are described by the same configuration block,
+which looks like this:
.. code:: yaml
- projects:
- some_project:
- remote-execution:
- execution-service:
- url: http://execution.some_project.example.com:50051
- instance-name: main
- storage-service:
- url: http://storage.some_project.example.com:11002
- instance-name: main
- action-cache-service:
- url: http://cache.some_project.example.com:50052
- instance-name: main
+ url: https://storage.fallback.example.com:11002
+ instance-name: main
+ auth:
+ server-cert: /keys/server.crt
+ client-cert: /keys/client.crt
+ client-key: /keys/client.key
+
+**Attributes:**
+
+* ``url``
+
+ Indicates the ``http`` or ``https`` url and optionally the port number of
+ where the service is located.
+
+* ``instance-name``
+
+ The instance name is optional. Instance names separate different shards on
+ the same endpoint (``url``). The instance name should be given to you by the
+ service provider of each service.
+
+ Not all service providers support instance names.
+
+* ``auth``
+ The :ref:`authentication attributes <config_remote_auth>` to connect to
+ this server.
+
+
+.. _user_config_project_overrides:
+
+Project specific value
+----------------------
+The ``projects`` key can be used to specify project specific configurations,
+the supported configurations on a project wide basis are listed here.
.. _user_config_strict_mode:
diff --git a/doc/source/using_configuring_cache_server.rst b/doc/source/using_configuring_cache_server.rst
index e5755a6..4e9b94d 100644
--- a/doc/source/using_configuring_cache_server.rst
+++ b/doc/source/using_configuring_cache_server.rst
@@ -13,41 +13,13 @@ In addition to the local caches, you can configure one or more remote caches and
BuildStream will then try to pull a suitable object from one of the remotes,
falling back to performing a local build or fetching a source if needed.
-Configuring BuildStream to use remote caches
---------------------------------------------
-A project will often set up continuous build infrastructure that pushes
-cached objects to a shared cache, so developers working on the project can
-make use of these pre-made objects instead of having to each build the whole
-project locally. The project can declare this cache in its
-project configuration file for :ref:`artifacts <project_essentials_artifacts>`
-and :ref:`sources <project_source_cache>`.
-
-Users can declare additional remote caches in the :ref:`user configuration
-<config_artifacts>`. There are several use cases for this: your project may not
-define its own cache, it may be useful to have a local mirror of its cache, or
-you may have a reason to share artifacts privately.
-
-Remote caches are identified by their URL. There are currently two supported
-protocols:
-
-* ``http``: Pull and push access, without transport-layer security
-* ``https``: Pull and push access, with transport-layer security
-
-BuildStream allows you to configure as many caches as you like, and will query
-them in a specific order:
-
-1. Project-specific overrides in the user config
-2. Project configuration
-3. User configuration
-
-When an an object is created locally, BuildStream will try to push it to all the
-caches which have the ``push: true`` flag set. You can also manually push
-artifacts to a specific cache using the :ref:`bst artifact push command
-<invoking_artifact_push>`.
-
-Objects are identified using the element or sources :ref:`cache key <cachekeys>`
-so the objects provided by a cache should be interchangable with those provided
-by any other cache.
+On the client side, cache servers are declared and configured in
+:ref:`user configuration <config_cache_servers>`, and since it is typical
+for projects to maintain their own cache servers, it is also possible for
+projects to provide recommended :ref:`artifact cache servers <project_artifact_cache>`
+and :ref:`source cache servers <project_source_cache>` through project
+configuration, so that downstream users can download from services
+provided by upstream projects by default.
Setting up a remote cache
@@ -173,7 +145,9 @@ Instance with push and requiring client authentication:
For this scenario, you can add the `--index-only` flag to the above
commands, and configure BuildStream to store artifact metadata and
files in a separate caches (e.g. bst-artifact-server and Buildbarn)
- using :ref:`"types" <project_essentials_split_artifacts>`.
+ using the ``type`` attribute of a :ref:`cache server <config_cache_servers>`
+ configuration.
+
Managing the cache with systemd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -260,14 +234,15 @@ We can then check if the services are successfully running with:
For more information on systemd services see:
`Creating Systemd Service Files <https://www.devdungeon.com/content/creating-systemd-service-files>`_.
+
Declaring remote caches
~~~~~~~~~~~~~~~~~~~~~~~
Remote caches can be declared within either:
-1. The project configuration for :ref:`artifact <project_essentials_artifacts>`
- and :ref:`sources <project_source_cache>`, or
-2. The user configuration for :ref:`artifacts <config_artifacts>` and
- :ref:`sources <config_sources>`.
+1. The user configuration for :ref:`artifacts <config_artifact_caches>` and
+ :ref:`sources <config_source_caches>`.
+2. The project configuration for :ref:`artifact <project_artifact_cache>`
+ and :ref:`sources <project_source_cache>`.
Please follow the above links to see examples showing how we declare remote
-caches in both the project configuration and the user configuration, respectively.
+caches in both user configuration the project configuration, respectively.
diff --git a/src/buildstream/plugins/elements/stack.py b/src/buildstream/plugins/elements/stack.py
index bd914ed..bdc9096 100644
--- a/src/buildstream/plugins/elements/stack.py
+++ b/src/buildstream/plugins/elements/stack.py
@@ -76,7 +76,7 @@ To accomplish this, you will need to know the cache key of the stack element
which was built remotely, possibly by inspecting the remote build log or by
deriving it with an equally configured BuildStream project, and you will
need read access to the artifact cache server which the build was uploaded to,
-this should be configured in your :ref:`user configuration file <config_artifacts>`.
+this should be configured in your :ref:`user configuration file <config_artifact_caches>`.
You can then checkout the remotely built stack using the
:ref:`bst artifact checkout <invoking_artifact_checkout>` command and providing