You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by GitBox <gi...@apache.org> on 2022/08/29 21:12:09 UTC
[GitHub] [tinkerpop] xiazcy opened a new pull request, #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
xiazcy opened a new pull request, #1797:
URL: https://github.com/apache/tinkerpop/pull/1797
Dockerized `gremlin-python` build and testing. Building, testing, and packaging of the python GLV are now executed inside of Docker, and python installation is no longer a system requirement. Outputs and distribution packages and are placed in the `target/` folder as before. Deployment of python still requires local Python installations.
Dockerized `gremlin-console` python-based integration tests. Python 3.8 is used for the image because the version of pytest used isn't compatible with later versions of Python. Building our own Docker image also allows for greater control over the version of Python and Java used.
The location of the python test output was changed in order to fit better with the bind mounted volume for Docker.
Minor documentation updates included. We will be looking at the Develop Docs more holistically for more appropriate updates in a separate PR.
Conflicts will be left alone for now, as we plan to rebased after the previous [PR](https://github.com/apache/tinkerpop/pull/1788) is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy commented on a diff in pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy commented on code in PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#discussion_r970117146
##########
gremlin-python/src/main/python/setup.py:
##########
@@ -75,7 +75,7 @@
tests_require=[
'pytest>=4.6.4,<7.2.0',
'mock>=3.0.5,<4.0.0',
- 'radish-bdd==0.8.6',
+ 'radish-bdd==0.13.4',
Review Comment:
Just to note that I've bumped the radish test suite to the latest version.
While it no longer matters inside of Docker (tests pass for either version), it resolves the error `Setup script exited with error in radish-parse_type setup command: use_2to3 is invalid.` for the previous pom set up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy commented on a diff in pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy commented on code in PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#discussion_r961926226
##########
gremlin-python/pom.xml:
##########
@@ -83,102 +83,31 @@ limitations under the License.
</activation>
<build>
<plugins>
- <!-- need to create python-reports directory at this point or else pytest can't write the report to it -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
+ <!-- copy source files in python directory to target/, to keep testing separate from packaging.
+ We use target/python3 for testing and target/python-packaged for distribution tasks.-->
<execution>
- <id>create-python-reports-directory</id>
- <phase>initialize</phase>
- <goals>
- <goal>run</goal>
- </goals>
- <configuration>
- <target>
- <mkdir dir="${project.build.directory}/python-reports"/>
- <mkdir dir="${project.build.directory}/python-packaged/env"/>
- </target>
- </configuration>
- </execution>
- <!-- copy files in python directory to target/py and run virtual env to sandbox python.
- there is no need to "activate" the virtualenv because all calls to python occur
- directly from bin/ -->
- <execution>
- <id>setup-py-env</id>
+ <id>setup-env</id>
<phase>process-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
- <!-- seems like we need a few different copies of the same source. all the
- different python stuff doesn't seem to want to share. we use
- /python3 for basic tests and stuff and /python-packaged for distribution
- tasks. some of the problem seems to stem from the python lifecycle not
- binding perfectly well to the maven lifecycle (integration tests seems to
- cause troubles specifically). note that the commands to install wheel are
- largely for safety in case someone is using an older version of virtualenv
- (which doesn't install wheel by default) -->
+ <!-- pytest will create python-reports inside target/python3/python-reports
+ as docker compose cannot access outside its working directory -->
<copy todir="${project.build.directory}/python3">
<fileset dir="src/main/python"/>
</copy>
- <exec dir="${project.build.directory}/python3" executable="python3"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
Review Comment:
That is one thing I wasn't sure of, how much the impact would be to release, not only with the set up removed, but also with the packaging taken inside of docker. I don't have any experiences with `pypi` releases though, I don't imagine there is any `mock release` to try? In any case, we could move the necessary python environment set up into the release profile to make sure it still works.
Some other options could be to move the packaging stage for python back into maven, so we keep the environment set up, to be sure it works the same way as before for package and release. Then down the road we could consider putting both package and release into Docker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy commented on pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy commented on PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#issuecomment-1230870245
Also, please note that the files changed will overlap with the previous [PR](https://github.com/apache/tinkerpop/pull/1788) as they share the same set up.
The PR-specific changes are in the `gremlin-python` and `gremlin-console` folders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] vkagamlyk commented on pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
vkagamlyk commented on PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#issuecomment-1234717343
Build/test time for GLV with and without Docker
Testing environment:
m4.xlarge AWS EC2 instance (4 vCPU, 16GiB RAM)
OS: Ubuntu 22.04.1 LTS
Java: openjdk version "11.0.16" 2022-07-19
OpenJDK Runtime Environment (build 11.0.16+8-post-Ubuntu-0ubuntu122.04)
OpenJDK 64-Bit Server VM (build 11.0.16+8-post-Ubuntu-0ubuntu122.04, mixed mode, sharing)
Docker: version 20.10.17, build 100c701
.glv files add with `touch {gremlin-python,gremlin-javascript,gremlin-go}/.glv`
Tests performed with command `sudo time mvn clean install`
First test run skipped.
| branch |.glv added | Average time |
| --- | --- | --- |
| 3.5 | no | 20:47.52 |
| 3.5 | yes | 24:56.99 |
| dockerize-others | yes | 28:55.19 |
Build time increase is ~15%
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] codecov-commenter commented on pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#issuecomment-1245884971
# [Codecov](https://codecov.io/gh/apache/tinkerpop/pull/1797?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#1797](https://codecov.io/gh/apache/tinkerpop/pull/1797?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (69bf61f) into [3.5-dev](https://codecov.io/gh/apache/tinkerpop/commit/33b60d9e34493bc04b7df45d0bd2c35080146238?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (33b60d9) will **increase** coverage by `0.03%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## 3.5-dev #1797 +/- ##
=============================================
+ Coverage 69.39% 69.42% +0.03%
- Complexity 8921 8925 +4
=============================================
Files 861 861
Lines 40858 40858
Branches 5384 5384
=============================================
+ Hits 28352 28366 +14
+ Misses 10602 10588 -14
Partials 1904 1904
```
| [Impacted Files](https://codecov.io/gh/apache/tinkerpop/pull/1797?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../traversal/step/map/PageRankVertexProgramStep.java](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS90aW5rZXJwb3AvZ3JlbWxpbi9wcm9jZXNzL2NvbXB1dGVyL3RyYXZlcnNhbC9zdGVwL21hcC9QYWdlUmFua1ZlcnRleFByb2dyYW1TdGVwLmphdmE=) | `72.09% <0.00%> (-4.66%)` | :arrow_down: |
| [...e/tinkerpop/gremlin/server/util/MetricManager.java](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3RpbmtlcnBvcC9ncmVtbGluL3NlcnZlci91dGlsL01ldHJpY01hbmFnZXIuamF2YQ==) | `51.93% <0.00%> (-0.56%)` | :arrow_down: |
| [...nkerpop/gremlin/groovy/engine/GremlinExecutor.java](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1ncm9vdnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3RpbmtlcnBvcC9ncmVtbGluL2dyb292eS9lbmdpbmUvR3JlbWxpbkV4ZWN1dG9yLmphdmE=) | `87.28% <0.00%> (-0.43%)` | :arrow_down: |
| [...in/process/traversal/dsl/graph/GraphTraversal.java](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS90aW5rZXJwb3AvZ3JlbWxpbi9wcm9jZXNzL3RyYXZlcnNhbC9kc2wvZ3JhcGgvR3JhcGhUcmF2ZXJzYWwuamF2YQ==) | `91.17% <0.00%> (+0.54%)` | :arrow_up: |
| [...apache/tinkerpop/gremlin/server/GremlinServer.java](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3RpbmtlcnBvcC9ncmVtbGluL3NlcnZlci9HcmVtbGluU2VydmVyLmphdmE=) | `57.07% <0.00%> (+1.46%)` | :arrow_up: |
| [gremlin-go/driver/gorillaTransporter.go](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1nby9kcml2ZXIvZ29yaWxsYVRyYW5zcG9ydGVyLmdv) | `66.98% <0.00%> (+2.83%)` | :arrow_up: |
| [...rg/apache/tinkerpop/gremlin/driver/Connection.java](https://codecov.io/gh/apache/tinkerpop/pull/1797/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z3JlbWxpbi1kcml2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3RpbmtlcnBvcC9ncmVtbGluL2RyaXZlci9Db25uZWN0aW9uLmphdmE=) | `64.18% <0.00%> (+6.08%)` | :arrow_up: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy commented on a diff in pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy commented on code in PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#discussion_r970121721
##########
gremlin-python/pom.xml:
##########
@@ -83,102 +83,31 @@ limitations under the License.
</activation>
<build>
<plugins>
- <!-- need to create python-reports directory at this point or else pytest can't write the report to it -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
+ <!-- copy source files in python directory to target/, to keep testing separate from packaging.
+ We use target/python3 for testing and target/python-packaged for distribution tasks.-->
<execution>
- <id>create-python-reports-directory</id>
- <phase>initialize</phase>
- <goals>
- <goal>run</goal>
- </goals>
- <configuration>
- <target>
- <mkdir dir="${project.build.directory}/python-reports"/>
- <mkdir dir="${project.build.directory}/python-packaged/env"/>
- </target>
- </configuration>
- </execution>
- <!-- copy files in python directory to target/py and run virtual env to sandbox python.
- there is no need to "activate" the virtualenv because all calls to python occur
- directly from bin/ -->
- <execution>
- <id>setup-py-env</id>
+ <id>setup-env</id>
<phase>process-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
- <!-- seems like we need a few different copies of the same source. all the
- different python stuff doesn't seem to want to share. we use
- /python3 for basic tests and stuff and /python-packaged for distribution
- tasks. some of the problem seems to stem from the python lifecycle not
- binding perfectly well to the maven lifecycle (integration tests seems to
- cause troubles specifically). note that the commands to install wheel are
- largely for safety in case someone is using an older version of virtualenv
- (which doesn't install wheel by default) -->
+ <!-- pytest will create python-reports inside target/python3/python-reports
+ as docker compose cannot access outside its working directory -->
<copy todir="${project.build.directory}/python3">
<fileset dir="src/main/python"/>
</copy>
- <exec dir="${project.build.directory}/python3" executable="python3"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
Review Comment:
The environment set up is added to the deploy profile in commit [ca0cdf0](https://github.com/apache/tinkerpop/pull/1797/commits/ca0cdf047710895f38a143fd0b9eada3604620f6). There could be issues we won't realized until the actual release process though (hopefully not).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy commented on pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy commented on PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#issuecomment-1247157925
VOTE +1. Given the two weeks review period, and that there are no additional comments, I will merge this through today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] spmallette commented on a diff in pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
spmallette commented on code in PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#discussion_r961877497
##########
gremlin-python/pom.xml:
##########
@@ -83,102 +83,31 @@ limitations under the License.
</activation>
<build>
<plugins>
- <!-- need to create python-reports directory at this point or else pytest can't write the report to it -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
+ <!-- copy source files in python directory to target/, to keep testing separate from packaging.
+ We use target/python3 for testing and target/python-packaged for distribution tasks.-->
<execution>
- <id>create-python-reports-directory</id>
- <phase>initialize</phase>
- <goals>
- <goal>run</goal>
- </goals>
- <configuration>
- <target>
- <mkdir dir="${project.build.directory}/python-reports"/>
- <mkdir dir="${project.build.directory}/python-packaged/env"/>
- </target>
- </configuration>
- </execution>
- <!-- copy files in python directory to target/py and run virtual env to sandbox python.
- there is no need to "activate" the virtualenv because all calls to python occur
- directly from bin/ -->
- <execution>
- <id>setup-py-env</id>
+ <id>setup-env</id>
<phase>process-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
- <!-- seems like we need a few different copies of the same source. all the
- different python stuff doesn't seem to want to share. we use
- /python3 for basic tests and stuff and /python-packaged for distribution
- tasks. some of the problem seems to stem from the python lifecycle not
- binding perfectly well to the maven lifecycle (integration tests seems to
- cause troubles specifically). note that the commands to install wheel are
- largely for safety in case someone is using an older version of virtualenv
- (which doesn't install wheel by default) -->
+ <!-- pytest will create python-reports inside target/python3/python-reports
+ as docker compose cannot access outside its working directory -->
<copy todir="${project.build.directory}/python3">
<fileset dir="src/main/python"/>
</copy>
- <exec dir="${project.build.directory}/python3" executable="python3"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
Review Comment:
as i sit here admiring all this pom.xml code getting simplified behind docker, i wonder if these changes will allow releases to actually work. for example, here we get rid of the virtualenv setup of python3, wheel, setuptools, etc. which would all be needed on release day (i think) when we deploy to pypi with `-Dpypi`: https://github.com/apache/tinkerpop/pull/1797/files#diff-924caaf2e60b71e4f805fe7d35f9c8895cb8d23ae258b12b7497b21d91ab639fR249-R269
thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy commented on a diff in pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy commented on code in PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#discussion_r961926226
##########
gremlin-python/pom.xml:
##########
@@ -83,102 +83,31 @@ limitations under the License.
</activation>
<build>
<plugins>
- <!-- need to create python-reports directory at this point or else pytest can't write the report to it -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
+ <!-- copy source files in python directory to target/, to keep testing separate from packaging.
+ We use target/python3 for testing and target/python-packaged for distribution tasks.-->
<execution>
- <id>create-python-reports-directory</id>
- <phase>initialize</phase>
- <goals>
- <goal>run</goal>
- </goals>
- <configuration>
- <target>
- <mkdir dir="${project.build.directory}/python-reports"/>
- <mkdir dir="${project.build.directory}/python-packaged/env"/>
- </target>
- </configuration>
- </execution>
- <!-- copy files in python directory to target/py and run virtual env to sandbox python.
- there is no need to "activate" the virtualenv because all calls to python occur
- directly from bin/ -->
- <execution>
- <id>setup-py-env</id>
+ <id>setup-env</id>
<phase>process-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
- <!-- seems like we need a few different copies of the same source. all the
- different python stuff doesn't seem to want to share. we use
- /python3 for basic tests and stuff and /python-packaged for distribution
- tasks. some of the problem seems to stem from the python lifecycle not
- binding perfectly well to the maven lifecycle (integration tests seems to
- cause troubles specifically). note that the commands to install wheel are
- largely for safety in case someone is using an older version of virtualenv
- (which doesn't install wheel by default) -->
+ <!-- pytest will create python-reports inside target/python3/python-reports
+ as docker compose cannot access outside its working directory -->
<copy todir="${project.build.directory}/python3">
<fileset dir="src/main/python"/>
</copy>
- <exec dir="${project.build.directory}/python3" executable="python3"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
Review Comment:
That is one thing I wasn't sure of, how much the impact would be to release, not only with the set up removed, but also with the packaging taken inside of docker. I don't have any experiences with `pypi` releases though, I don't imagine there is any `mock release` to try? In any case, we could move the necessary python environment set up into the release profile to make sure it still works.
Some other options could be to also move the packaging stage for python back into maven, so we keep the environment set up, to be sure it works the same way as before for package and release. Then down the road we could consider putting both package and release into Docker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] spmallette commented on a diff in pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
spmallette commented on code in PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797#discussion_r961944489
##########
gremlin-python/pom.xml:
##########
@@ -83,102 +83,31 @@ limitations under the License.
</activation>
<build>
<plugins>
- <!-- need to create python-reports directory at this point or else pytest can't write the report to it -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
+ <!-- copy source files in python directory to target/, to keep testing separate from packaging.
+ We use target/python3 for testing and target/python-packaged for distribution tasks.-->
<execution>
- <id>create-python-reports-directory</id>
- <phase>initialize</phase>
- <goals>
- <goal>run</goal>
- </goals>
- <configuration>
- <target>
- <mkdir dir="${project.build.directory}/python-reports"/>
- <mkdir dir="${project.build.directory}/python-packaged/env"/>
- </target>
- </configuration>
- </execution>
- <!-- copy files in python directory to target/py and run virtual env to sandbox python.
- there is no need to "activate" the virtualenv because all calls to python occur
- directly from bin/ -->
- <execution>
- <id>setup-py-env</id>
+ <id>setup-env</id>
<phase>process-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
- <!-- seems like we need a few different copies of the same source. all the
- different python stuff doesn't seem to want to share. we use
- /python3 for basic tests and stuff and /python-packaged for distribution
- tasks. some of the problem seems to stem from the python lifecycle not
- binding perfectly well to the maven lifecycle (integration tests seems to
- cause troubles specifically). note that the commands to install wheel are
- largely for safety in case someone is using an older version of virtualenv
- (which doesn't install wheel by default) -->
+ <!-- pytest will create python-reports inside target/python3/python-reports
+ as docker compose cannot access outside its working directory -->
<copy todir="${project.build.directory}/python3">
<fileset dir="src/main/python"/>
</copy>
- <exec dir="${project.build.directory}/python3" executable="python3"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
- failonerror="true">
- <arg line="--version"/>
- </exec>
- <exec dir="${project.build.directory}/python3" executable="virtualenv"
Review Comment:
> In any case, we could move the necessary python environment set up into the release profile to make sure it still works.
i was thinking you should just do this for now. in the slightly longer term (i.e. next release manager tries this approach out and documents it), maybe we just go to native release instructions.
> Then down the road we could consider putting both package and release into Docker.
yes - that's a good long term goal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tinkerpop] xiazcy merged pull request #1797: Dev Environments: Dockerize Testing for Python and Gremlin Console (Python-based tests only)
Posted by GitBox <gi...@apache.org>.
xiazcy merged PR #1797:
URL: https://github.com/apache/tinkerpop/pull/1797
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org