You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@buildstream.apache.org by tv...@apache.org on 2021/02/04 08:16:08 UTC
[buildstream] branch jennis/push_pull_artifacts created (now
5b06e2d)
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a change to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git.
at 5b06e2d WIP: cli.py: Spawn pull jobs in another process to isolate gRPC calls
This branch includes the following new commits:
new 94b309c Move push and pull to the new artifact subcommand group
new aceb399 cli: Split classify_artifacts helper up
new cb6d1c4 artifactcache.py: Add new method ArtifactCache.create_remote()
new f609f0a artifactcache.py: Allow ArtifactCache to handle pulling/pushing refs
new a790b2f artifactcache.py: API Cleanup - Add contains_ref() method
new 7043109 WIP: Ensure push/pull can deal with artifact refs
new d3f1a4d cli.py: Handle the pulling of an artifact's buildtree
new 6e12c3f cli.py: Handle the --remote option for pulling artifacts
new 5b06e2d WIP: cli.py: Spawn pull jobs in another process to isolate gRPC calls
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[buildstream] 08/09: cli.py: Handle the --remote option for pulling
artifacts
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 6e12c3fe5d1067b97e1850355f4e69cc6905f8d4
Author: James Ennis <ja...@codethink.com>
AuthorDate: Tue Jan 15 12:52:49 2019 +0000
cli.py: Handle the --remote option for pulling artifacts
---
buildstream/_frontend/cli.py | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/buildstream/_frontend/cli.py b/buildstream/_frontend/cli.py
index 985f5d2..24012ba 100644
--- a/buildstream/_frontend/cli.py
+++ b/buildstream/_frontend/cli.py
@@ -1001,6 +1001,18 @@ def artifact_pull(app, artifacts, deps, remote):
remotes = usr_remotes + project_remotes
+ if remote: # Check if this is one of the available remotes
+ # Here we are restricting the user to only use a remote that is defined
+ # in either the project.conf or the user config, should the user be able to *try*
+ # and pull from any specified remote?
+ remote_found = False
+ for spec in remotes:
+ if remote == spec.url:
+ remotes = [spec]
+ remote_found = True
+ if not remote_found:
+ raise AppError("Remote: '{}' not found.".format(remote))
+
# Pull buildtrees?
excluded_subdirs = ["buildtree"] if app.context.pull_buildtrees else None
[buildstream] 04/09: artifactcache.py: Allow ArtifactCache to
handle pulling/pushing refs
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit f609f0a6debd19658498d11cfa13bd597e8900b1
Author: James Ennis <ja...@codethink.com>
AuthorDate: Tue Jan 15 11:17:52 2019 +0000
artifactcache.py: Allow ArtifactCache to handle pulling/pushing refs
With the addition of the artifact subcommands, which are able to
process artifact refs, we should not have to rely on CASCache
internals to handle artifacts. This commit ensures that the
pulling/pushing of elements (and their cache keys) is handled
by ArtifactCache.
---
buildstream/_artifactcache/artifactcache.py | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/buildstream/_artifactcache/artifactcache.py b/buildstream/_artifactcache/artifactcache.py
index af27a1e..383d173 100644
--- a/buildstream/_artifactcache/artifactcache.py
+++ b/buildstream/_artifactcache/artifactcache.py
@@ -661,7 +661,7 @@ class ArtifactCache():
display_key = element._get_brief_display_key()
element.status("Pulling artifact {} <- {}".format(display_key, remote.spec.url))
- if self.cas.pull(ref, remote, progress=progress, subdir=subdir, excluded_subdirs=excluded_subdirs):
+ if self.pull_ref(ref, remote, progress=progress, subdir=subdir, excluded_subdirs=excluded_subdirs):
element.info("Pulled artifact {} <- {}".format(display_key, remote.spec.url))
if subdir:
# Attempt to extract subdir into artifact extract dir if it already exists
@@ -681,6 +681,22 @@ class ArtifactCache():
return False
+ # pull_ref():
+ #
+ # Pull artifact from one of the configured remote repositories.
+ #
+ # Args:
+ # ref (str): The ref to pull
+ # progress (callable): The progress callback, if any
+ # subdir (str): The optional specific subdir to pull
+ # excluded_subdirs (list): The optional list of subdirs to not pull
+ #
+ # Returns:
+ # (bool): True if pull was successful, False if artifact was not available
+ #
+ def pull_ref(self, ref, remote, *, progress=None, subdir=None, excluded_subdirs=None):
+ return self.cas.pull(ref, remote, progress=progress, subdir=subdir, excluded_subdirs=excluded_subdirs)
+
# pull_tree():
#
# Pull a single Tree rather than an artifact.
[buildstream] 09/09: WIP: cli.py: Spawn pull jobs in another
process to isolate gRPC calls
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 5b06e2dd0a77a5faf2c52558d825ed52d3cb6ed0
Author: James Ennis <ja...@codethink.com>
AuthorDate: Tue Jan 15 15:07:15 2019 +0000
WIP: cli.py: Spawn pull jobs in another process to isolate gRPC calls
---
buildstream/_frontend/cli.py | 59 ++++++++++++++++++++++++++++++++++++++------
1 file changed, 51 insertions(+), 8 deletions(-)
diff --git a/buildstream/_frontend/cli.py b/buildstream/_frontend/cli.py
index 24012ba..1752b8f 100644
--- a/buildstream/_frontend/cli.py
+++ b/buildstream/_frontend/cli.py
@@ -1,4 +1,6 @@
+import multiprocessing
import os
+import signal
import sys
from contextlib import ExitStack
from fnmatch import fnmatch
@@ -973,6 +975,9 @@ def artifact_pull(app, artifacts, deps, remote):
none: No dependencies, just the element itself
all: All dependencies
"""
+ from .. import _signals
+ from .. import utils
+ from .._exceptions import CASError
with app.initialized(session_name="Pull"):
cache = app.context.artifactcache
@@ -1016,14 +1021,52 @@ def artifact_pull(app, artifacts, deps, remote):
# Pull buildtrees?
excluded_subdirs = ["buildtree"] if app.context.pull_buildtrees else None
- # Try to pull the artifact from one of the remotes
- remotes = [cache.create_remote(spec) for spec in remotes]
- for ref in artifacts:
- if cache.contains_ref(ref):
- continue
- for remote in remotes:
- if cache.pull_ref(ref, remote, exclude_subdirs=exclude_subdirs):
- break
+ # Define function to be used by multiprocessing
+ def _pull_artifact(artifact_cache_obj, remote_specs, refs, excluded_subdirs, queue):
+ remotes = [artifact_cache_obj.create_remote(spec) for spec in remote_specs]
+ pull_attempts = {'skipped': [], 'pulled': [], 'failed': []}
+ for ref in refs:
+ if artifact_cache_obj.contains_ref(ref):
+ pull_attempts['skipped'].append(ref)
+ continue
+
+ pulled = False
+ for remote in remotes:
+ try:
+ if artifact_cache_obj.pull_ref(ref, remote, excluded_subdirs=excluded_subdirs):
+ pull_attempts['pulled'].append((ref, remote.spec.url))
+ break
+ except CASError as e:
+ queue.put(e)
+ raise
+ if not pulled:
+ pull_attempts['failed'].append(ref)
+
+ queue.put(pull_attempts)
+
+ q = multiprocessing.Queue()
+ p = multiprocessing.Process(target=_pull_artifact, args=(cache, remotes, artifacts, excluded_subdirs, q))
+ try:
+ with _signals.blocked([signal.SIGINT], ignore=False):
+ p.start()
+
+ result = q.get()
+ p.join()
+ except KeyboardInterrupt:
+ utils._kill_process_tree(p.pid)
+ raise
+
+ # Output to user
+ if isinstance(result, dict):
+ for ref in result['skipped']:
+ click.echo("'{}' already available - pull skipped.".format(ref))
+ for ref, remote in result['pulled']:
+ click.echo("'{}' pulled from '{}'.".format(ref, remote))
+ for ref in result['failed']:
+ urls = [remote.url for remote in remotes]
+ click.echo("'{}' not available in remotes: {}".format(ref, urls)) # Ugly list print
+ else:
+ raise result
##################################################################
[buildstream] 01/09: Move push and pull to the new artifact
subcommand group
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 94b309c323ecaefb28ac787604a0a35e899d48de
Author: James Ennis <ja...@codethink.com>
AuthorDate: Mon Jan 7 18:00:54 2019 +0000
Move push and pull to the new artifact subcommand group
This commit also ensures that if we try to use the 'old' commands,
BuildStream will fail and instruct the user to use the new command.
---
buildstream/_frontend/cli.py | 178 ++++++++++++++++++++--------------
tests/artifactcache/config.py | 2 +-
tests/artifactcache/junctions.py | 4 +-
tests/completions/completions.py | 14 +--
tests/frontend/help.py | 3 +-
tests/frontend/pull.py | 12 +--
tests/frontend/push.py | 16 +--
tests/frontend/workspace.py | 4 +-
tests/integration/build-tree.py | 5 +-
tests/integration/pullbuildtrees.py | 34 +++----
tests/sandboxes/remote-exec-config.py | 4 +-
11 files changed, 156 insertions(+), 120 deletions(-)
diff --git a/buildstream/_frontend/cli.py b/buildstream/_frontend/cli.py
index bdcf3ca..e8e4c8b 100644
--- a/buildstream/_frontend/cli.py
+++ b/buildstream/_frontend/cli.py
@@ -356,78 +356,6 @@ def build(app, elements, all_, track_, track_save, track_all, track_except, trac
##################################################################
-# Pull Command #
-##################################################################
-@cli.command(short_help="Pull a built artifact")
-@click.option('--deps', '-d', default='none',
- type=click.Choice(['none', 'all']),
- help='The dependency artifacts to pull (default: none)')
-@click.option('--remote', '-r',
- help="The URL of the remote cache (defaults to the first configured cache)")
-@click.argument('elements', nargs=-1,
- type=click.Path(readable=False))
-@click.pass_obj
-def pull(app, elements, deps, remote):
- """Pull a built artifact from the configured remote artifact cache.
-
- By default the artifact will be pulled one of the configured caches
- if possible, following the usual priority order. If the `--remote` flag
- is given, only the specified cache will be queried.
-
- Specify `--deps` to control which artifacts to pull:
-
- \b
- none: No dependencies, just the element itself
- all: All dependencies
- """
-
- with app.initialized(session_name="Pull"):
- if not elements:
- guessed_target = app.context.guess_element()
- if guessed_target:
- elements = (guessed_target,)
-
- app.stream.pull(elements, selection=deps, remote=remote)
-
-
-##################################################################
-# Push Command #
-##################################################################
-@cli.command(short_help="Push a built artifact")
-@click.option('--deps', '-d', default='none',
- type=click.Choice(['none', 'all']),
- help='The dependencies to push (default: none)')
-@click.option('--remote', '-r', default=None,
- help="The URL of the remote cache (defaults to the first configured cache)")
-@click.argument('elements', nargs=-1,
- type=click.Path(readable=False))
-@click.pass_obj
-def push(app, elements, deps, remote):
- """Push a built artifact to a remote artifact cache.
-
- The default destination is the highest priority configured cache. You can
- override this by passing a different cache URL with the `--remote` flag.
-
- If bst has been configured to include build trees on artifact pulls,
- an attempt will be made to pull any required build trees to avoid the
- skipping of partial artifacts being pushed.
-
- Specify `--deps` to control which artifacts to push:
-
- \b
- none: No dependencies, just the element itself
- all: All dependencies
- """
- with app.initialized(session_name="Push"):
- if not elements:
- guessed_target = app.context.guess_element()
- if guessed_target:
- elements = (guessed_target,)
-
- app.stream.push(elements, selection=deps, remote=remote)
-
-
-##################################################################
# Show Command #
##################################################################
@cli.command(short_help="Show elements in the pipeline")
@@ -1011,6 +939,78 @@ def artifact():
################################################################
+# Artifact Pull Command #
+################################################################
+@artifact.command(name="pull", short_help="Pull a built artifact")
+@click.option('--deps', '-d', default='none',
+ type=click.Choice(['none', 'all']),
+ help='The dependency artifacts to pull (default: none)')
+@click.option('--remote', '-r',
+ help="The URL of the remote cache (defaults to the first configured cache)")
+@click.argument('elements', nargs=-1,
+ type=click.Path(readable=False))
+@click.pass_obj
+def artifact_pull(app, elements, deps, remote):
+ """Pull a built artifact from the configured remote artifact cache.
+
+ By default the artifact will be pulled one of the configured caches
+ if possible, following the usual priority order. If the `--remote` flag
+ is given, only the specified cache will be queried.
+
+ Specify `--deps` to control which artifacts to pull:
+
+ \b
+ none: No dependencies, just the element itself
+ all: All dependencies
+ """
+
+ with app.initialized(session_name="Pull"):
+ if not elements:
+ guessed_target = app.context.guess_element()
+ if guessed_target:
+ elements = (guessed_target,)
+
+ app.stream.pull(elements, selection=deps, remote=remote)
+
+
+##################################################################
+# Artifact Push Command #
+##################################################################
+@artifact.command(name="push", short_help="Push a built artifact")
+@click.option('--deps', '-d', default='none',
+ type=click.Choice(['none', 'all']),
+ help='The dependencies to push (default: none)')
+@click.option('--remote', '-r', default=None,
+ help="The URL of the remote cache (defaults to the first configured cache)")
+@click.argument('elements', nargs=-1,
+ type=click.Path(readable=False))
+@click.pass_obj
+def artifact_push(app, elements, deps, remote):
+ """Push a built artifact to a remote artifact cache.
+
+ The default destination is the highest priority configured cache. You can
+ override this by passing a different cache URL with the `--remote` flag.
+
+ If bst has been configured to include build trees on artifact pulls,
+ an attempt will be made to pull any required build trees to avoid the
+ skipping of partial artifacts being pushed.
+
+ Specify `--deps` to control which artifacts to push:
+
+ \b
+ none: No dependencies, just the element itself
+ all: All dependencies
+ """
+ with app.initialized(session_name="Push"):
+ if not elements:
+ guessed_target = app.context.guess_element()
+ if guessed_target:
+ elements = (guessed_target,)
+
+ app.stream.push(elements, selection=deps, remote=remote)
+
+
+################################################################
# Artifact Log Command #
################################################################
@artifact.command(name='log', short_help="Show logs of an artifact")
@@ -1116,3 +1116,37 @@ def fetch(app, elements, deps, track_, except_, track_cross_junctions):
def track(app, elements, deps, except_, cross_junctions):
click.echo("This command is now obsolete. Use `bst source track` instead.", err=True)
sys.exit(1)
+
+
+################################################################
+# Pull Command #
+################################################################
+@cli.command(short_help="Pull a built artifact", hidden=True)
+@click.option('--deps', '-d', default='none',
+ type=click.Choice(['none', 'all']),
+ help='The dependency artifacts to pull (default: none)')
+@click.option('--remote', '-r',
+ help="The URL of the remote cache (defaults to the first configured cache)")
+@click.argument('elements', nargs=-1,
+ type=click.Path(readable=False))
+@click.pass_obj
+def pull(app, elements, deps, remote):
+ click.echo("This command is now obsolete. Use `bst artifact pull` instead.", err=True)
+ sys.exit(1)
+
+
+##################################################################
+# Push Command #
+##################################################################
+@cli.command(short_help="Push a built artifact", hidden=True)
+@click.option('--deps', '-d', default='none',
+ type=click.Choice(['none', 'all']),
+ help='The dependencies to push (default: none)')
+@click.option('--remote', '-r', default=None,
+ help="The URL of the remote cache (defaults to the first configured cache)")
+@click.argument('elements', nargs=-1,
+ type=click.Path(readable=False))
+@click.pass_obj
+def push(app, elements, deps, remote):
+ click.echo("This command is now obsolete. Use `bst artifact push` instead.", err=True)
+ sys.exit(1)
diff --git a/tests/artifactcache/config.py b/tests/artifactcache/config.py
index df40d10..bd7d119 100644
--- a/tests/artifactcache/config.py
+++ b/tests/artifactcache/config.py
@@ -138,5 +138,5 @@ def test_missing_certs(cli, datafiles, config_key, config_value):
# Use `pull` here to ensure we try to initialize the remotes, triggering the error
#
# This does not happen for a simple `bst show`.
- result = cli.run(project=project, args=['pull', 'element.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', 'element.bst'])
result.assert_main_error(ErrorDomain.LOAD, LoadErrorReason.INVALID_DATA)
diff --git a/tests/artifactcache/junctions.py b/tests/artifactcache/junctions.py
index 378d007..c6d6921 100644
--- a/tests/artifactcache/junctions.py
+++ b/tests/artifactcache/junctions.py
@@ -58,7 +58,7 @@ def test_push_pull(cli, tmpdir, datafiles):
project_set_artifacts(base_project, base_share.repo)
# Now try bst push
- result = cli.run(project=project, args=['push', '--deps', 'all', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', '--deps', 'all', 'target.bst'])
assert result.exit_code == 0
# And finally assert that the artifacts are in the right shares
@@ -78,7 +78,7 @@ def test_push_pull(cli, tmpdir, datafiles):
assert state != 'cached'
# Now try bst pull
- result = cli.run(project=project, args=['pull', '--deps', 'all', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', '--deps', 'all', 'target.bst'])
assert result.exit_code == 0
# And assert that they are again in the local cache, without having built
diff --git a/tests/completions/completions.py b/tests/completions/completions.py
index 372ed78..53d0e97 100644
--- a/tests/completions/completions.py
+++ b/tests/completions/completions.py
@@ -11,8 +11,6 @@ MAIN_COMMANDS = [
'checkout ',
'help ',
'init ',
- 'pull ',
- 'push ',
'shell ',
'show ',
'source ',
@@ -54,6 +52,12 @@ SOURCE_COMMANDS = [
'track ',
]
+ARTIFACT_COMMANDS = [
+ 'push ',
+ 'pull ',
+ 'log ',
+]
+
WORKSPACE_COMMANDS = [
'close ',
'list ',
@@ -117,8 +121,7 @@ def assert_completion_failed(cli, cmd, word_idx, expected, cwd=None):
@pytest.mark.parametrize("cmd,word_idx,expected", [
('bst', 0, []),
('bst ', 1, MAIN_COMMANDS),
- ('bst pu', 1, ['pull ', 'push ']),
- ('bst pul', 1, ['pull ']),
+ ('bst artifact ', 2, ARTIFACT_COMMANDS),
('bst source ', 2, SOURCE_COMMANDS),
('bst w ', 1, ['workspace ']),
('bst workspace ', 2, WORKSPACE_COMMANDS),
@@ -272,9 +275,8 @@ def test_argument_element_invalid(datafiles, cli, project, cmd, word_idx, expect
@pytest.mark.parametrize("cmd,word_idx,expected", [
('bst he', 1, ['help ']),
('bst help ', 2, MAIN_COMMANDS),
+ ('bst help artifact ', 3, ARTIFACT_COMMANDS),
('bst help in', 2, ['init ']),
- ('bst help p', 2, ['pull ', 'push ']),
- ('bst help p', 2, ['pull ', 'push ']),
('bst help source ', 3, SOURCE_COMMANDS),
('bst help w', 2, ['workspace ']),
('bst help workspace ', 3, WORKSPACE_COMMANDS),
diff --git a/tests/frontend/help.py b/tests/frontend/help.py
index e3b406f..331eb43 100644
--- a/tests/frontend/help.py
+++ b/tests/frontend/help.py
@@ -18,10 +18,9 @@ def test_help_main(cli):
@pytest.mark.parametrize("command", [
+ ('artifact'),
('build'),
('checkout'),
- ('pull'),
- ('push'),
('shell'),
('show'),
('source'),
diff --git a/tests/frontend/pull.py b/tests/frontend/pull.py
index c883e20..254b6d5 100644
--- a/tests/frontend/pull.py
+++ b/tests/frontend/pull.py
@@ -70,7 +70,7 @@ def test_push_pull_all(cli, tmpdir, datafiles):
assert cli.get_element_state(project, element_name) != 'cached'
# Now try bst pull
- result = cli.run(project=project, args=['pull', '--deps', 'all', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', '--deps', 'all', 'target.bst'])
result.assert_success()
# And assert that it's again in the local cache, without having built
@@ -111,7 +111,7 @@ def test_pull_secondary_cache(cli, tmpdir, datafiles):
assert cli.get_element_state(project, 'target.bst') != 'cached'
# Now try bst pull
- result = cli.run(project=project, args=['pull', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', 'target.bst'])
result.assert_success()
# And assert that it's again in the local cache, without having built,
@@ -146,7 +146,7 @@ def test_push_pull_specific_remote(cli, tmpdir, datafiles):
# Now try `bst push` to the good_share.
result = cli.run(project=project, args=[
- 'push', 'target.bst', '--remote', good_share.repo
+ 'artifact', 'push', 'target.bst', '--remote', good_share.repo
])
result.assert_success()
@@ -161,7 +161,7 @@ def test_push_pull_specific_remote(cli, tmpdir, datafiles):
artifacts = os.path.join(cli.directory, 'artifacts')
shutil.rmtree(artifacts)
- result = cli.run(project=project, args=['pull', 'target.bst', '--remote',
+ result = cli.run(project=project, args=['artifact', 'pull', 'target.bst', '--remote',
good_share.repo])
result.assert_success()
@@ -216,7 +216,7 @@ def test_push_pull_non_strict(cli, tmpdir, datafiles):
assert cli.get_element_state(project, 'target.bst') == 'waiting'
# Now try bst pull
- result = cli.run(project=project, args=['pull', '--deps', 'all', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', '--deps', 'all', 'target.bst'])
result.assert_success()
# And assert that the target is again in the local cache, without having built
@@ -291,7 +291,7 @@ def test_push_pull_cross_junction(cli, tmpdir, datafiles):
assert cli.get_element_state(project, 'junction.bst:import-etc.bst') == 'buildable'
# Now try bst pull
- result = cli.run(project=project, args=['pull', 'junction.bst:import-etc.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', 'junction.bst:import-etc.bst'])
result.assert_success()
# And assert that it's again in the local cache, without having built
diff --git a/tests/frontend/push.py b/tests/frontend/push.py
index 153d433..6ed163a 100644
--- a/tests/frontend/push.py
+++ b/tests/frontend/push.py
@@ -82,7 +82,7 @@ def test_push(cli, tmpdir, datafiles):
with create_artifact_share(os.path.join(str(tmpdir), 'artifactshare2')) as share2:
# Try pushing with no remotes configured. This should fail.
- result = cli.run(project=project, args=['push', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', 'target.bst'])
result.assert_main_error(ErrorDomain.STREAM, None)
# Configure bst to pull but not push from a cache and run `bst push`.
@@ -90,7 +90,7 @@ def test_push(cli, tmpdir, datafiles):
cli.configure({
'artifacts': {'url': share1.repo, 'push': False},
})
- result = cli.run(project=project, args=['push', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', 'target.bst'])
result.assert_main_error(ErrorDomain.STREAM, None)
# Configure bst to push to one of the caches and run `bst push`. This works.
@@ -100,7 +100,7 @@ def test_push(cli, tmpdir, datafiles):
{'url': share2.repo, 'push': True},
]
})
- result = cli.run(project=project, args=['push', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', 'target.bst'])
assert_not_shared(cli, share1, project, 'target.bst')
assert_shared(cli, share2, project, 'target.bst')
@@ -114,7 +114,7 @@ def test_push(cli, tmpdir, datafiles):
{'url': share2.repo, 'push': True},
]
})
- result = cli.run(project=project, args=['push', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', 'target.bst'])
assert_shared(cli, share1, project, 'target.bst')
assert_shared(cli, share2, project, 'target.bst')
@@ -156,7 +156,7 @@ def test_push_all(cli, tmpdir, datafiles):
# Now try bst push all the deps
result = cli.run(project=project, args=[
- 'push', 'target.bst',
+ 'artifact', 'push', 'target.bst',
'--deps', 'all'
])
result.assert_success()
@@ -346,7 +346,7 @@ def test_recently_pulled_artifact_does_not_expire(cli, datafiles, tmpdir):
assert cli.get_element_state(project, 'element1.bst') != 'cached'
# Pull the element1 from the remote cache (this should update its mtime)
- result = cli.run(project=project, args=['pull', 'element1.bst', '--remote',
+ result = cli.run(project=project, args=['artifact', 'pull', 'element1.bst', '--remote',
share.repo])
result.assert_success()
@@ -386,7 +386,7 @@ def test_push_cross_junction(cli, tmpdir, datafiles):
cli.configure({
'artifacts': {'url': share.repo, 'push': True},
})
- result = cli.run(project=project, args=['push', 'junction.bst:import-etc.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', 'junction.bst:import-etc.bst'])
cache_key = cli.get_element_key(project, 'junction.bst:import-etc.bst')
assert share.has_artifact('subtest', 'import-etc.bst', cache_key)
@@ -407,7 +407,7 @@ def test_push_already_cached(caplog, cli, tmpdir, datafiles):
result.assert_success()
assert "SKIPPED Push" not in result.stderr
- result = cli.run(project=project, args=['push', 'target.bst'])
+ result = cli.run(project=project, args=['artifact', 'push', 'target.bst'])
result.assert_success()
assert not result.get_pushed_elements(), "No elements should have been pushed since the cache was populated"
diff --git a/tests/frontend/workspace.py b/tests/frontend/workspace.py
index 00c0bd8..912cc9f 100644
--- a/tests/frontend/workspace.py
+++ b/tests/frontend/workspace.py
@@ -1105,10 +1105,10 @@ def test_external_push_pull(cli, datafiles, tmpdir_factory, guess_element):
'artifacts': {'url': share.repo, 'push': True}
})
- result = cli.run(project=project, args=['-C', workspace, 'push'] + arg_elm)
+ result = cli.run(project=project, args=['-C', workspace, 'artifact', 'push'] + arg_elm)
result.assert_success()
- result = cli.run(project=project, args=['-C', workspace, 'pull', '--deps', 'all'] + arg_elm)
+ result = cli.run(project=project, args=['-C', workspace, 'artifact', 'pull', '--deps', 'all'] + arg_elm)
result.assert_success()
diff --git a/tests/integration/build-tree.py b/tests/integration/build-tree.py
index 8c0628d..b50d841 100644
--- a/tests/integration/build-tree.py
+++ b/tests/integration/build-tree.py
@@ -130,7 +130,8 @@ def test_buildtree_pulled(cli, tmpdir, datafiles):
assert cli.get_element_state(project, element_name) != 'cached'
# Pull from cache, ensuring cli options is set to pull the buildtree
- result = cli.run(project=project, args=['--pull-buildtrees', 'pull', '--deps', 'all', element_name])
+ result = cli.run(project=project,
+ args=['--pull-buildtrees', 'artifact', 'pull', '--deps', 'all', element_name])
result.assert_success()
# Check it's using the cached build tree
@@ -164,7 +165,7 @@ def test_buildtree_options(cli, tmpdir, datafiles):
assert cli.get_element_state(project, element_name) != 'cached'
# Pull from cache, but do not include buildtrees.
- result = cli.run(project=project, args=['pull', '--deps', 'all', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', '--deps', 'all', element_name])
result.assert_success()
# The above is the simplest way I know to create a local cache without any buildtrees.
diff --git a/tests/integration/pullbuildtrees.py b/tests/integration/pullbuildtrees.py
index f6fc712..89710a7 100644
--- a/tests/integration/pullbuildtrees.py
+++ b/tests/integration/pullbuildtrees.py
@@ -55,12 +55,12 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# Pull artifact with default config, assert that pulling again
# doesn't create a pull job, then assert with buildtrees user
# config set creates a pull job.
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name not in result.get_pulled_elements()
cli.configure({'cache': {'pull-buildtrees': True}})
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
default_state(cli, tmpdir, share1)
@@ -68,13 +68,13 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# with buildtrees cli flag set creates a pull job.
# Also assert that the buildtree is added to the artifact's
# extract dir
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
elementdigest = share1.has_artifact('test', element_name, cli.get_element_key(project, element_name))
buildtreedir = os.path.join(str(tmpdir), 'artifacts', 'extract', 'test', 'autotools-amhello',
elementdigest.hash, 'buildtree')
assert not os.path.isdir(buildtreedir)
- result = cli.run(project=project, args=['--pull-buildtrees', 'pull', element_name])
+ result = cli.run(project=project, args=['--pull-buildtrees', 'artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
assert os.path.isdir(buildtreedir)
default_state(cli, tmpdir, share1)
@@ -83,21 +83,21 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# that pulling with the same user config doesn't creates a pull job,
# or when buildtrees cli flag is set.
cli.configure({'cache': {'pull-buildtrees': True}})
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name not in result.get_pulled_elements()
- result = cli.run(project=project, args=['--pull-buildtrees', 'pull', element_name])
+ result = cli.run(project=project, args=['--pull-buildtrees', 'artifact', 'pull', element_name])
assert element_name not in result.get_pulled_elements()
default_state(cli, tmpdir, share1)
# Pull artifact with default config and buildtrees cli flag set, then assert
# that pulling with pullbuildtrees set in user config doesn't create a pull
# job.
- result = cli.run(project=project, args=['--pull-buildtrees', 'pull', element_name])
+ result = cli.run(project=project, args=['--pull-buildtrees', 'artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
cli.configure({'cache': {'pull-buildtrees': True}})
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name not in result.get_pulled_elements()
default_state(cli, tmpdir, share1)
@@ -105,10 +105,10 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# can't be pushed to an artifact share, then assert that a complete build element
# can be. This will attempt a partial pull from share1 and then a partial push
# to share2
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
cli.configure({'artifacts': {'url': share2.repo, 'push': True}})
- result = cli.run(project=project, args=['push', element_name])
+ result = cli.run(project=project, args=['artifact', 'push', element_name])
assert element_name not in result.get_pushed_elements()
assert not share2.has_artifact('test', element_name, cli.get_element_key(project, element_name))
@@ -116,10 +116,10 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# successfully pushed to the remote. This will attempt to pull the buildtree
# from share1 and then a 'complete' push to share2
cli.configure({'artifacts': {'url': share1.repo, 'push': False}})
- result = cli.run(project=project, args=['--pull-buildtrees', 'pull', element_name])
+ result = cli.run(project=project, args=['--pull-buildtrees', 'artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
cli.configure({'artifacts': {'url': share2.repo, 'push': True}})
- result = cli.run(project=project, args=['push', element_name])
+ result = cli.run(project=project, args=['artifact', 'push', element_name])
assert element_name in result.get_pushed_elements()
assert share2.has_artifact('test', element_name, cli.get_element_key(project, element_name))
default_state(cli, tmpdir, share1)
@@ -128,10 +128,10 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# if pull-buildtrees is set, however as share3 is the only defined remote and is empty,
# assert that no element artifact buildtrees are pulled (no available remote buildtree) and thus the
# artifact cannot be pushed.
- result = cli.run(project=project, args=['pull', element_name])
+ result = cli.run(project=project, args=['artifact', 'pull', element_name])
assert element_name in result.get_pulled_elements()
cli.configure({'artifacts': {'url': share3.repo, 'push': True}})
- result = cli.run(project=project, args=['--pull-buildtrees', 'push', element_name])
+ result = cli.run(project=project, args=['--pull-buildtrees', 'artifact', 'push', element_name])
assert "Attempting to fetch missing artifact buildtrees" in result.stderr
assert element_name not in result.get_pulled_elements()
assert not os.path.isdir(buildtreedir)
@@ -143,7 +143,7 @@ def test_pullbuildtrees(cli, tmpdir, datafiles, integration_cache):
# to the empty share3. This gives the ability to attempt push currently partial artifacts to a remote,
# without exlipictly requiring a bst pull.
cli.configure({'artifacts': [{'url': share1.repo, 'push': False}, {'url': share3.repo, 'push': True}]})
- result = cli.run(project=project, args=['--pull-buildtrees', 'push', element_name])
+ result = cli.run(project=project, args=['--pull-buildtrees', 'artifact', 'push', element_name])
assert "Attempting to fetch missing artifact buildtrees" in result.stderr
assert element_name in result.get_pulled_elements()
assert os.path.isdir(buildtreedir)
diff --git a/tests/sandboxes/remote-exec-config.py b/tests/sandboxes/remote-exec-config.py
index 31cb072..9cf0a8f 100644
--- a/tests/sandboxes/remote-exec-config.py
+++ b/tests/sandboxes/remote-exec-config.py
@@ -42,7 +42,7 @@ def test_old_and_new_configs(cli, datafiles):
# Use `pull` here to ensure we try to initialize the remotes, triggering the error
#
# This does not happen for a simple `bst show`.
- result = cli.run(project=project, args=['pull', 'element.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', 'element.bst'])
result.assert_main_error(ErrorDomain.LOAD, LoadErrorReason.INVALID_DATA, "specify one")
@@ -97,5 +97,5 @@ def test_empty_config(cli, datafiles):
# Use `pull` here to ensure we try to initialize the remotes, triggering the error
#
# This does not happen for a simple `bst show`.
- result = cli.run(project=project, args=['pull', 'element.bst'])
+ result = cli.run(project=project, args=['artifact', 'pull', 'element.bst'])
result.assert_main_error(ErrorDomain.LOAD, LoadErrorReason.INVALID_DATA, "specify one")
[buildstream] 06/09: WIP: Ensure push/pull can deal with artifact
refs
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit 70431096b764b9bb6a047e0aa8634a94b8b4ddf7
Author: James Ennis <ja...@codethink.com>
AuthorDate: Wed Jan 9 10:02:02 2019 +0000
WIP: Ensure push/pull can deal with artifact refs
---
buildstream/_frontend/cli.py | 61 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 51 insertions(+), 10 deletions(-)
diff --git a/buildstream/_frontend/cli.py b/buildstream/_frontend/cli.py
index e3196ea..41dee45 100644
--- a/buildstream/_frontend/cli.py
+++ b/buildstream/_frontend/cli.py
@@ -944,7 +944,6 @@ def _classify_artifacts(names, cas, project_directory):
return targets, refs
-
@cli.group(short_help="Manipulate cached artifacts")
def artifact():
"""Manipulate cached artifacts"""
@@ -959,10 +958,9 @@ def artifact():
help='The dependency artifacts to pull (default: none)')
@click.option('--remote', '-r',
help="The URL of the remote cache (defaults to the first configured cache)")
-@click.argument('elements', nargs=-1,
- type=click.Path(readable=False))
+@click.argument('artifacts', type=click.Path(), nargs=-1)
@click.pass_obj
-def artifact_pull(app, elements, deps, remote):
+def artifact_pull(app, artifacts, deps, remote):
"""Pull a built artifact from the configured remote artifact cache.
By default the artifact will be pulled one of the configured caches
@@ -977,12 +975,40 @@ def artifact_pull(app, elements, deps, remote):
"""
with app.initialized(session_name="Pull"):
- if not elements:
+ cache = app.context.artifactcache
+
+ elements, artifacts = _classify_artifacts(artifacts, cache.cas,
+ app.project.directory)
+
+ # Guess the element if we're in a workspace
+ if not elements and not artifacts:
guessed_target = app.context.guess_element()
if guessed_target:
elements = (guessed_target,)
- app.stream.pull(elements, selection=deps, remote=remote)
+ if artifacts and deps != 'none':
+ raise AppError("--deps may not be used with artifact refs")
+
+ if elements:
+ app.stream.pull(elements, selection=deps, remote=remote)
+ if artifacts:
+ # Determine all available remotes
+ usr_remotes = app.context.artifact_cache_specs # list of CASRemoteSpec objects
+
+ project = app.context.get_toplevel_project() # Should I get all projects here...?
+ project.ensure_fully_loaded()
+ project_remotes = project.artifact_cache_specs # list of CASRemoteSpec objects
+
+ remotes = usr_remotes + project_remotes
+
+ # Try to pull the artifact from one of the remotes
+ remotes = [cache.create_remote(spec) for spec in remotes]
+ for ref in artifacts:
+ if cache.contains_ref(ref):
+ continue
+ for remote in remotes:
+ if cache.pull_ref(ref, remote):
+ break
##################################################################
@@ -994,10 +1020,9 @@ def artifact_pull(app, elements, deps, remote):
help='The dependencies to push (default: none)')
@click.option('--remote', '-r', default=None,
help="The URL of the remote cache (defaults to the first configured cache)")
-@click.argument('elements', nargs=-1,
- type=click.Path(readable=False))
+@click.argument('artifacts', type=click.Path(), nargs=-1)
@click.pass_obj
-def artifact_push(app, elements, deps, remote):
+def artifact_push(app, artifacts, deps, remote):
"""Push a built artifact to a remote artifact cache.
The default destination is the highest priority configured cache. You can
@@ -1014,12 +1039,28 @@ def artifact_push(app, elements, deps, remote):
all: All dependencies
"""
with app.initialized(session_name="Push"):
+ cache = app.context.artifactcache
+
+ elements, artifacts = _classify_artifacts(artifacts, cache.cas,
+ app.project.directory)
+
+ # Guess the element if we're in a workspace
if not elements:
guessed_target = app.context.guess_element()
if guessed_target:
elements = (guessed_target,)
- app.stream.push(elements, selection=deps, remote=remote)
+ if artifacts and deps != 'none':
+ raise AppError("--deps may not be used with artifact refs")
+
+ if elements:
+ app.stream.push(elements, selection=deps, remote=remote)
+ # FIXME: We can only obtain project/user config through the stream API,
+ # which we need to determine the remote in order for pull to pull from.
+ # We can't just go straight to artifactcache here. Thus Stream.Pull()
+ # will fail because it expects a list of element names (.bst).
+ if artifacts:
+ app.stream.push(artifacts, selection='none', remote=remote)
################################################################
[buildstream] 02/09: cli: Split classify_artifacts helper up
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit aceb3995e84fdbfa2adc6f9e4f61c0ce5a4e1679
Author: Richard Maw <ri...@codethink.co.uk>
AuthorDate: Wed Dec 12 18:23:19 2018 +0000
cli: Split classify_artifacts helper up
Classifying artifact refs is useful for glob expansion
even in cases where an element target name would be inappropriate.
---
buildstream/_frontend/cli.py | 52 +++++++++++++++++++++++++++-----------------
1 file changed, 32 insertions(+), 20 deletions(-)
diff --git a/buildstream/_frontend/cli.py b/buildstream/_frontend/cli.py
index e8e4c8b..e3196ea 100644
--- a/buildstream/_frontend/cli.py
+++ b/buildstream/_frontend/cli.py
@@ -901,36 +901,48 @@ def workspace_list(app):
#############################################################
# Artifact Commands #
#############################################################
-def _classify_artifacts(names, cas, project_directory):
- element_targets = []
- artifact_refs = []
- element_globs = []
- artifact_globs = []
-
+def _classify_element_targets(names, project_directory):
+ globs = []
+ targets = []
+ unmatched = []
for name in names:
if name.endswith('.bst'):
if any(c in "*?[" for c in name):
- element_globs.append(name)
+ globs.append(name)
else:
- element_targets.append(name)
+ targets.append(name)
else:
- if any(c in "*?[" for c in name):
- artifact_globs.append(name)
- else:
- artifact_refs.append(name)
+ unmatched.append(name)
- if element_globs:
+ if globs:
for dirpath, _, filenames in os.walk(project_directory):
for filename in filenames:
- element_path = os.path.join(dirpath, filename).lstrip(project_directory).lstrip('/')
- if any(fnmatch(element_path, glob) for glob in element_globs):
- element_targets.append(element_path)
+ element_path = os.path.relpath(os.path.join(dirpath, filename), start=project_directory)
+ if any(fnmatch(element_path, glob) for glob in globs):
+ targets.append(element_path)
+ return targets, unmatched
+
+
+def _classify_artifact_refs(names, cas):
+ globs = []
+ refs = []
+ for name in names:
+ if any(c in "*?[" for c in name):
+ globs.append(name)
+ else:
+ refs.append(name)
+ if globs:
+ refs.extend(ref for ref in cas.list_refs()
+ if any(fnmatch(ref, glob) for glob in globs))
+ return refs
+
+
+def _classify_artifacts(names, cas, project_directory):
+ targets, unmatched = _classify_element_targets(names, project_directory)
+ refs = _classify_artifact_refs(unmatched, cas)
- if artifact_globs:
- artifact_refs.extend(ref for ref in cas.list_refs()
- if any(fnmatch(ref, glob) for glob in artifact_globs))
+ return targets, refs
- return element_targets, artifact_refs
@cli.group(short_help="Manipulate cached artifacts")
[buildstream] 03/09: artifactcache.py: Add new method
ArtifactCache.create_remote()
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit cb6d1c46b9c17d06fb0c4f307e1e459bf0def6c1
Author: James Ennis <ja...@codethink.com>
AuthorDate: Tue Jan 15 11:16:35 2019 +0000
artifactcache.py: Add new method ArtifactCache.create_remote()
This allows for the creation of a CASRemote object using the
ArtifactCache API
---
buildstream/_artifactcache/artifactcache.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/buildstream/_artifactcache/artifactcache.py b/buildstream/_artifactcache/artifactcache.py
index b4b8df3..af27a1e 100644
--- a/buildstream/_artifactcache/artifactcache.py
+++ b/buildstream/_artifactcache/artifactcache.py
@@ -399,7 +399,7 @@ class ArtifactCache():
if remote_spec.push:
self._has_push_remotes = True
- remotes[remote_spec.url] = CASRemote(remote_spec)
+ remotes[remote_spec.url] = self.create_remote(remote_spec)
for project in self.context.get_projects():
remote_specs = self.global_remote_specs
@@ -419,6 +419,9 @@ class ArtifactCache():
self._remotes[project] = project_remotes
+ def create_remote(self, remote_spec):
+ return CASRemote(remote_spec)
+
# contains():
#
# Check whether the artifact for the specified Element is already available
[buildstream] 07/09: cli.py: Handle the pulling of an artifact's
buildtree
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit d3f1a4d82c65a79cc7bd1c97e79034922e3ffc05
Author: James Ennis <ja...@codethink.com>
AuthorDate: Tue Jan 15 16:42:33 2019 +0000
cli.py: Handle the pulling of an artifact's buildtree
---
buildstream/_frontend/cli.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/buildstream/_frontend/cli.py b/buildstream/_frontend/cli.py
index 41dee45..985f5d2 100644
--- a/buildstream/_frontend/cli.py
+++ b/buildstream/_frontend/cli.py
@@ -1001,13 +1001,16 @@ def artifact_pull(app, artifacts, deps, remote):
remotes = usr_remotes + project_remotes
+ # Pull buildtrees?
+ excluded_subdirs = ["buildtree"] if app.context.pull_buildtrees else None
+
# Try to pull the artifact from one of the remotes
remotes = [cache.create_remote(spec) for spec in remotes]
for ref in artifacts:
if cache.contains_ref(ref):
continue
for remote in remotes:
- if cache.pull_ref(ref, remote):
+ if cache.pull_ref(ref, remote, exclude_subdirs=exclude_subdirs):
break
[buildstream] 05/09: artifactcache.py: API Cleanup - Add
contains_ref() method
Posted by tv...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tvb pushed a commit to branch jennis/push_pull_artifacts
in repository https://gitbox.apache.org/repos/asf/buildstream.git
commit a790b2f3cbf1280980bb95acab9f009d86284640
Author: James Ennis <ja...@codethink.com>
AuthorDate: Fri Jan 11 15:49:48 2019 +0000
artifactcache.py: API Cleanup - Add contains_ref() method
This ensures that we can determine whether an artifact is locally cached
(as well as an element) using the ArtifactCache API.
---
buildstream/_artifactcache/artifactcache.py | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/buildstream/_artifactcache/artifactcache.py b/buildstream/_artifactcache/artifactcache.py
index 383d173..14d40b7 100644
--- a/buildstream/_artifactcache/artifactcache.py
+++ b/buildstream/_artifactcache/artifactcache.py
@@ -424,18 +424,30 @@ class ArtifactCache():
# contains():
#
- # Check whether the artifact for the specified Element is already available
- # in the local artifact cache.
+ # Check whether the (project state) artifact of the specified Element is
+ # already available in the local artifact cache.
#
# Args:
# element (Element): The Element to check
# key (str): The cache key to use
#
- # Returns: True if the artifact is in the cache, False otherwise
+ # Returns: True if the Element's (project state) artifact is in the cache,
+ # False otherwise
#
def contains(self, element, key):
ref = self.get_artifact_fullname(element, key)
+ return self.contains_ref(ref)
+ # contains_ref():
+ #
+ # Check whether an artifact is already available in the local artifact cache.
+ #
+ # Args:
+ # ref (str): The ref to check
+ #
+ # Returns: True if the artifact is in the cache, False otherwise
+ #
+ def contains_ref(self, ref):
return self.cas.contains(ref)
# contains_subdir_artifact():