You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@buildstream.apache.org by GitBox <gi...@apache.org> on 2022/08/24 09:37:35 UTC
[GitHub] [buildstream] ssssam opened a new issue, #1740: Intermittent test failure in test_shell_pull_cached_buildtree
ssssam opened a new issue, #1740:
URL: https://github.com/apache/buildstream/issues/1740
The test test_shell_pull_cached_buildtree sometimes fails.
Example log: https://gitlab.com/samthursfield/buildstream-docker-images/-/jobs/2923053985
Failure info displayed by pytest:
```
___________ test_shell_pull_cached_buildtree[pull-without-buildtree] ___________
share_with_buildtrees = <tests.testutils.artifactshare.ArtifactShare object at 0x7fcb3d6fb3d0>
datafiles = local('/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2')
cli = <buildstream._testing.runcli.Cli object at 0x7fcb3c85ead0>
pull_deps = 'all', pull_buildtree = False
@pytest.mark.datafiles(DATA_DIR)
@pytest.mark.skipif(not HAVE_SANDBOX, reason="Only available with a functioning sandbox")
@pytest.mark.parametrize(
"pull_deps,pull_buildtree",
[
# Don't pull at all
(None, False),
# Pull only dependencies
("build", False),
# Pull all elements including the shell element, but without the buildtree
("all", False),
# Pull all elements including the shell element, and pull buildtrees
("all", True),
],
ids=["no-pull", "pull-only-deps", "pull-without-buildtree", "pull-with-buildtree"],
)
def test_shell_pull_cached_buildtree(share_with_buildtrees, datafiles, cli, pull_deps, pull_buildtree):
project = str(datafiles)
element_name = "build-shell/buildtree.bst"
cli.configure({"artifacts": {"servers": [{"url": share_with_buildtrees.repo}]}})
# Optionally pull the buildtree along with `bst artifact pull`
maybe_pull_deps(cli, project, element_name, pull_deps, pull_buildtree)
# Run the shell and request that required artifacts and buildtrees should be pulled
result = cli.run(
project=project,
args=[
"--pull-buildtrees",
"shell",
"--build",
element_name,
"--use-buildtree",
"--",
"cat",
"test",
],
)
# In this case, we should succeed every time, regardless of what was
# originally available in the local cache.
#
> result.assert_success()
tests/integration/shellbuildtrees.py:325:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <buildstream._testing.runcli.Result object at 0x7fcb3c74e3b0>
fail_message = ''
def assert_success(self, fail_message=""):
> assert self.exit_code == 0, fail_message
E AssertionError
.tox/py310-nocover/lib64/python3.10/site-packages/buildstream/_testing/runcli.py:107: AssertionError
----------------------------- Captured stdout call -----------------------------
BuildStream exited with code 0 for invocation:
bst --no-colors --config /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf --directory /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2 artifact pull --deps all build-shell/buildtree.bst
Program stderr was:
[--:--:--][ ][ main:core activity ] START Pull
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
3 subtasks processed
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
3 of 3 subtasks processed
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
3 of 3 subtasks processed
BuildStream Version 1.95.2+4.gb6b96a6b1
Session Start: Tuesday, 23-08-2022 at 17:12:11
Project: test (/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2)
Targets: build-shell/buildtree.bst
User Configuration
Configuration File: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf
Cache Directory: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache
Log Files: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/logs
Source Mirrors: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/sources
Build Area: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/build
Strict Build Plan: Yes
Maximum Fetch Tasks: 10
Maximum Build Tasks: 4
Maximum Push Tasks: 4
Maximum Network Retries: 2
Project: test
Project Options
arch: x86-64
linux: 1
Element Plugins
manual: core plugin
stack: core plugin
import: core plugin
Source Plugins
tar: core plugin
Artifact cache servers
* http://localhost:44409/
push: False type: all instance: None
loaded from: /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf [line 3 column 4]
Pipeline
fetch needed 4ed9f63d0077a00322df997adda1c3eb96d9142cd2848074b3325f6f6f9e530e base/base-alpine.bst
waiting 024ff07b5cef16947a874b815f01ed5afbe338d2beb69634e9d9f3813efb5782 base.bst
waiting 4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099 build-shell/buildtree.bst
===============================================================================
[--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] START test/base-base-alpine/4ed9f63d-pull.261.log
[--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] STATUS Pulling artifact 4ed9f63d <- http://localhost:44409/
[--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] STATUS Pulling data for artifact 4ed9f63d <- http://localhost:44409/
[--:--:--][024ff07b][ pull:base.bst ] START test/base/024ff07b-pull.261.log
[--:--:--][024ff07b][ pull:base.bst ] STATUS Pulling artifact 024ff07b <- http://localhost:44409/
[--:--:--][024ff07b][ pull:base.bst ] STATUS Pulling data for artifact 024ff07b <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] START test/build-shell-buildtree/4a47c98a-pull.261.log
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling artifact 4a47c98a <- http://localhost:44409/
[--:--:--][024ff07b][ pull:base.bst ] INFO Pulled artifact 024ff07b <- http://localhost:44409/
[00:00:00][024ff07b][ pull:base.bst ] SUCCESS test/base/024ff07b-pull.261.log
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling data for artifact 4a47c98a <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] INFO Pulled artifact 4a47c98a <- http://localhost:44409/
[00:00:00][4a47c98a][ pull:build-shell/buildtree.bst ] SUCCESS test/build-shell-buildtree/4a47c98a-pull.261.log
[--:--:--][4ed9f63d][ pull:base/base-alpine.bst ] INFO Pulled artifact 4ed9f63d <- http://localhost:44409/
[00:00:03][4ed9f63d][ pull:base/base-alpine.bst ] SUCCESS test/base-base-alpine/4ed9f63d-pull.261.log
[00:00:03][ ][ main:core activity ] SUCCESS Pull
Pipeline Summary
Total: 3
Session: 3
Pull Queue: processed 3, skipped 0, failed 0
BuildStream exited with code -1 for invocation:
bst --no-colors --config /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/buildstream.conf --directory /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2 --pull-buildtrees shell --build build-shell/buildtree.bst --use-buildtree -- cat test
Program stderr was:
[--:--:--][ ][ main:core activity ] START Loading elements
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
3 subtasks processed
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
3 of 3 subtasks processed
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
3 of 3 subtasks processed
[--:--:--][ ][ main:core activity ] INFO Attempting to fetch missing or incomplete artifact(s)
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] START test/build-shell-buildtree/4a47c98a-pull.261.log
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling artifact 4a47c98a <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling data for artifact 4a47c98a <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] START test/build-shell-buildtree/4a47c98a-pull.261.log
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling artifact 4a47c98a <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] STATUS Pulling data for artifact 4a47c98a <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] INFO Pulled artifact 4a47c98a <- http://localhost:44409/
[--:--:--][4a47c98a][ pull:build-shell/buildtree.bst ] INFO Pulled artifact 4a47c98a <- http://localhost:44409/
[00:00:00][4a47c98a][ pull:build-shell/buildtree.bst ] SUCCESS test/build-shell-buildtree/4a47c98a-pull.261.log
[00:00:00][4a47c98a][ pull:build-shell/buildtree.bst ] FAILURE Failed to link '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099 -> /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1': [Errno 17] File exists: '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099' -> '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1'
Printing the last 20 lines from log file:
/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/logs/test/build-shell-buildtree/4a47c98a-pull.261.log
======================================================================
BuildStream 1.95.2+4.gb6b96a6b1 - Tuesday, 23-08-2022 at 17:12:11
[--:--:--] START [4a47c98a] build-shell/buildtree.bst: Pull
[--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling artifact 4a47c98a <- http://localhost:44409/
[--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling data for artifact 4a47c98a <- http://localhost:44409/
[--:--:--] INFO [4a47c98a] build-shell/buildtree.bst: Pulled artifact 4a47c98a <- http://localhost:44409/
[00:00:00] SUCCESS [4a47c98a] build-shell/buildtree.bst: Pull
BuildStream 1.95.2+4.gb6b96a6b1 - Tuesday, 23-08-2022 at 17:12:14
[--:--:--] START [4a47c98a] build-shell/buildtree.bst: Pull
[--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling artifact 4a47c98a <- http://localhost:44409/
[--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling data for artifact 4a47c98a <- http://localhost:44409/
BuildStream 1.95.2+4.gb6b96a6b1 - Tuesday, 23-08-2022 at 17:12:14
[--:--:--] START [4a47c98a] build-shell/buildtree.bst: Pull
[--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling artifact 4a47c98a <- http://localhost:44409/
[--:--:--] STATUS [4a47c98a] build-shell/buildtree.bst: Pulling data for artifact 4a47c98a <- http://localhost:44409/
[--:--:--] INFO [4a47c98a] build-shell/buildtree.bst: Pulled artifact 4a47c98a <- http://localhost:44409/
[--:--:--] INFO [4a47c98a] build-shell/buildtree.bst: Pulled artifact 4a47c98a <- http://localhost:44409/
[00:00:00] SUCCESS [4a47c98a] build-shell/buildtree.bst: Pull
[00:00:00] FAILURE [4a47c98a] build-shell/buildtree.bst: Failed to link '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099 -> /home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1': [Errno 17] File exists: '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/4a47c98a10df39e65e99d471f96edc5b58d4ea5b9b1f221d0be832a8124b8099' -> '/home/testuser/buildstream/.tox/py310-nocover/tmp/test_shell_pull_cached_buildtr2/cache/artifacts/refs/test/build-shell-buildtree/38b730aec720dbddba843c9e3ce1ad8073e39d16cc4d25f752bc89d867fbaec1'
======================================================================
Error launching shell:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] gtristan commented on issue #1740: Intermittent test failure in test_shell_pull_cached_buildtree
Posted by "gtristan (via GitHub)" <gi...@apache.org>.
gtristan commented on issue #1740:
URL: https://github.com/apache/buildstream/issues/1740#issuecomment-1454617079
Reproduced today: https://github.com/apache/buildstream/actions/runs/4329430098/jobs/7560573504
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] gtristan commented on issue #1740: Intermittent test failure in test_shell_pull_cached_buildtree
Posted by GitBox <gi...@apache.org>.
gtristan commented on issue #1740:
URL: https://github.com/apache/buildstream/issues/1740#issuecomment-1226436068
I spent the greater part of Sunday fixing this, it was reproducible locally with `while tox — testname; do echo “success”; done`
Although I would get errors where the regex match count was 5 instead of the expected 6, this looks like a different error ?
see recent MRs to find my patch which adds a lock around the message cache in app.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] ssssam commented on issue #1740: Intermittent test failure in test_shell_pull_cached_buildtree
Posted by GitBox <gi...@apache.org>.
ssssam commented on issue #1740:
URL: https://github.com/apache/buildstream/issues/1740#issuecomment-1225542958
The above test run was using shared GitLab runners, it's possible that there's a filesystem precision issue at work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org