You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ariatosca.apache.org by tliron <gi...@git.apache.org> on 2017/07/10 15:09:16 UTC
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
GitHub user tliron opened a pull request:
https://github.com/apache/incubator-ariatosca/pull/183
ARIA-76 Parallelize PyTest
Use the PyTest xdist plugin to parallelize tests in boxed subprocesses.
Through benchmarking we discovered that paralellizing on the number of
CPU cores ("-n auto") provides the best times.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-ariatosca ARIA-76-parallel-pytest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-ariatosca/pull/183.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #183
----
commit b2310536e00894ec781cd12ee20a620257cbb437
Author: Tal Liron <ta...@gmail.com>
Date: 2017-07-10T15:06:05Z
ARIA-76 Parallelize PyTest
Use the PyTest xdist plugin to parallelize tests in boxed subprocesses.
Through benchmarking we discovered that paralellizing on the number of
CPU cores ("-n auto") provides the best times.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by tliron <gi...@git.apache.org>.
Github user tliron commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126884207
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126697248
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
can we add some documentation, since it looks kinda awkward
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126699227
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
can't we just export this as environment variable?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by tliron <gi...@git.apache.org>.
Github user tliron commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126884450
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
Yes AppVeyor and yes concurrency. :) AppVeyor works fine with the `-n auto` defaults, likely because it's running in a restricted VM. The problem is only with Travis, because the OS reports a very large amount of CPUs (12 in our case) but the container is extremely restricted, and thus we get failures.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca issue #183: ARIA-76 Parallelize PyTest
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-ariatosca/pull/183
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca issue #183: ARIA-76 Parallelize PyTest
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-ariatosca/pull/183
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126696990
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
What about appveyor? no concurrency?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by tliron <gi...@git.apache.org>.
Github user tliron closed the pull request at:
https://github.com/apache/incubator-ariatosca/pull/183
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-ariatosca/pull/183
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by tliron <gi...@git.apache.org>.
GitHub user tliron reopened a pull request:
https://github.com/apache/incubator-ariatosca/pull/183
ARIA-76 Parallelize PyTest
Use the PyTest xdist plugin to parallelize tests in boxed subprocesses.
Through benchmarking we discovered that paralellizing on the number of
CPU cores ("-n auto") provides the best times.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-ariatosca ARIA-76-parallel-pytest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-ariatosca/pull/183.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #183
----
commit b2310536e00894ec781cd12ee20a620257cbb437
Author: Tal Liron <ta...@gmail.com>
Date: 2017-07-10T15:06:05Z
ARIA-76 Parallelize PyTest
Use the PyTest xdist plugin to parallelize tests in boxed subprocesses.
Through benchmarking we discovered that paralellizing on the number of
CPU cores ("-n auto") provides the best times.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by tliron <gi...@git.apache.org>.
Github user tliron commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126884604
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
-1, because actually the name of the xdist argument is `--numprocesses`. Actually, it makes sense: the way xdist works is exactly by creating boxed subprocesses.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by mxmrlv <gi...@git.apache.org>.
Github user mxmrlv commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126696279
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
rename to PYTEST_CONCURRENCY
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
Posted by tliron <gi...@git.apache.org>.
Github user tliron commented on a diff in the pull request:
https://github.com/apache/incubator-ariatosca/pull/183#discussion_r126884186
--- Diff: .travis.yml ---
@@ -23,11 +23,12 @@ env:
- TOX_ENV=py26e2e
- TOX_ENV=py27ssh
- TOX_ENV=py26ssh
+- TOX_ENV=docs
install:
- pip install --upgrade pip
- pip install --upgrade setuptools
- pip install tox
script:
- pip --version
- tox --version
- - tox -e $TOX_ENV
+ - PYTEST_PROCESSES=1 tox -e $TOX_ENV
--- End diff --
No, there's is no script context here, it's one command at a time. :(
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---