[jira] [Comment Edited] (BEAM-2813) error: option --test-pipeline-options not recognized

["Mark Liu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-2813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144500#comment-16144500 ] 

Mark Liu edited comment on BEAM-2813 at 8/28/17 10:58 PM:
----------------------------------------------------------

I think this is a version mismatch in dependencies. This line may related to the actual error:
{code}
/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (pyasn1 0.3.3 (/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/lib/python2.7/site-packages), Requirement.parse('pyasn1==0.3.2'), set(['pyasn1-modules']))
RuntimeWarning)
{code}

Both pyasn1 and pyasn1-modules are indirect dependencies and used in many GCP packages. Two days ago, pyasn1 was updated to 0.3.3. However, pyasn1-modules(0.0.11) only requires pyasn1 0.3.2. I guess when nose upload customized plugin, it also examined the existing packages and found versions conflict.

Yesterday, pyasn1-modules was updated to [0.1.1|https://pypi.python.org/pypi/pyasn1-modules] which fix this problem by pointing to latest pyasn1 (0.3.3). So Jenkins started passing.


was (Author: markflyhigh):
I think this is a version mismatch in dependencies. This line may related to the actual error:
{code}
/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (pyasn1 0.3.3 (/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/lib/python2.7/site-packages), Requirement.parse('pyasn1==0.3.2'), set(['pyasn1-modules']))
RuntimeWarning)
{code}

Both pyasn1 and pyasn1-modules are indirect dependencies for many GCP packages. Two days ago, pyasn1 was updated to 0.3.3. However, pyasn1-modules(0.0.11) only requires pyasn1 0.3.2. I guess when nose upload customized plugin, it also examined the existing packages and found versions conflict.

Yesterday, pyasn1-modules was updated to [0.1.1|https://pypi.python.org/pypi/pyasn1-modules] which fix this problem by pointing to latest pyasn1 (0.3.3). So Jenkins started passing.

> error: option --test-pipeline-options not recognized
> ----------------------------------------------------
>
>                 Key: BEAM-2813
>                 URL: https://issues.apache.org/jira/browse/BEAM-2813
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py
>            Reporter: Ahmet Altay
>            Assignee: Mark Liu
>
> Python post commits 3004 to 3008 (all 5) failed with this error, but somehow fixed in 3009. Mark do you know what might be causing this?
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Python_Verify/3004/
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Python_Verify/3009/
> The error is:
> # Run ValidatesRunner tests on Google Cloud Dataflow service
> echo ">>> RUNNING DATAFLOW RUNNER VALIDATESRUNNER TESTS"
> >>> RUNNING DATAFLOW RUNNER VALIDATESRUNNER TESTS
> python setup.py nosetests \
>   --attr ValidatesRunner \
>   --nocapture \
>   --processes=4 \
>   --process-timeout=900 \
>   --test-pipeline-options=" \
>     --runner=TestDataflowRunner \
>     --project=$PROJECT \
>     --staging_location=$GCS_LOCATION/staging-validatesrunner-test \
>     --temp_location=$GCS_LOCATION/temp-validatesrunner-test \
>     --sdk_location=$SDK_LOCATION \
>     --requirements_file=postcommit_requirements.txt \
>     --num_workers=1"
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py:341: UserWarning: Normalizing '2.2.0.dev' to '2.2.0.dev0'
>   normalized_version,
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (pyasn1 0.3.3 (/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/lib/python2.7/site-packages), Requirement.parse('pyasn1==0.3.2'), set(['pyasn1-modules']))
>   RuntimeWarning)
> usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
>    or: setup.py --help [cmd1 cmd2 ...]
>    or: setup.py --help-commands
>    or: setup.py cmd --help
> error: option --test-pipeline-options not recognized



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)