You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/23 18:07:20 UTC
[GitHub] [beam] KevinGG opened a new pull request #11507: [BEAM-7923] Support docker screendiff integration tests
KevinGG opened a new pull request #11507:
URL: https://github.com/apache/beam/pull/11507
1. Added 2 more chrome options so that the interactive notebook screendiff integration tests work within docker.
2. Updated the golden screenshot.
Details:
1. By default, Docker runs a container with a /dev/shm shared memory space 64MB. This is typically too small for Chrome and will cause Chrome to crash when rendering large pages. To fix, run the container with docker run --shm-size=1gb to increase the size of /dev/shm. Since Chrome 65, this is no longer necessary. Instead, launch the browser with the --disable-dev-shm-usage flag
2. Without the chrome options, a test running within docker container will fail with following error:
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620791900
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] KevinGG edited a comment on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
KevinGG edited a comment on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620175112
The error seems to be affecting multiple [precommits](https://builds.apache.org/job/beam_PreCommit_Python_Commit/).
Created https://issues.apache.org/jira/browse/BEAM-9832 to track the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620123842
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620745665
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620734469
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] KevinGG commented on issue #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
KevinGG commented on issue #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-618559403
R: @aaltay
PTAL, thx!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620734399
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on a change in pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11507:
URL: https://github.com/apache/beam/pull/11507#discussion_r414820395
##########
File path: sdks/python/setup.py
##########
@@ -229,6 +229,8 @@ def get_version():
'selenium>=3.141.0,<4',
'needle>=0.5.0,<1',
'chromedriver-binary>=80,<81',
+ # use a fixed version of PIL for different python versions
+ 'pillow==7.1.1',
Review comment:
why?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] KevinGG commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
KevinGG commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620175112
The error seems to be affecting multiple precommits.
Created https://issues.apache.org/jira/browse/BEAM-9832 to track the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620123998
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-619206222
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620745290
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] KevinGG commented on a change in pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
KevinGG commented on a change in pull request #11507:
URL: https://github.com/apache/beam/pull/11507#discussion_r414821611
##########
File path: sdks/python/setup.py
##########
@@ -229,6 +229,8 @@ def get_version():
'selenium>=3.141.0,<4',
'needle>=0.5.0,<1',
'chromedriver-binary>=80,<81',
+ # use a fixed version of PIL for different python versions
+ 'pillow==7.1.1',
Review comment:
Python comes with PIL, but different versions of Python has different PIL versions.
To rule out the difference when taking screenshot, we use a fixed version of PIL explicitly for testing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-620733719
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on a change in pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11507:
URL: https://github.com/apache/beam/pull/11507#discussion_r414847501
##########
File path: sdks/python/setup.py
##########
@@ -229,6 +229,8 @@ def get_version():
'selenium>=3.141.0,<4',
'needle>=0.5.0,<1',
'chromedriver-binary>=80,<81',
+ # use a fixed version of PIL for different python versions
+ 'pillow==7.1.1',
Review comment:
Do we need to pin it? Pinning usually results in other issues. Maybe less so here because it is a test dependency.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on issue #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
aaltay commented on issue #11507:
URL: https://github.com/apache/beam/pull/11507#issuecomment-618563856
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] KevinGG commented on a change in pull request #11507: [BEAM-7923] Support docker screendiff integration tests
Posted by GitBox <gi...@apache.org>.
KevinGG commented on a change in pull request #11507:
URL: https://github.com/apache/beam/pull/11507#discussion_r415999888
##########
File path: sdks/python/setup.py
##########
@@ -229,6 +229,8 @@ def get_version():
'selenium>=3.141.0,<4',
'needle>=0.5.0,<1',
'chromedriver-binary>=80,<81',
+ # use a fixed version of PIL for different python versions
+ 'pillow==7.1.1',
Review comment:
I've made it any version under a major version >=7,<8.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org