[jira] [Commented] (BEAM-5788) wordcount_fnapi_it failed on TestDataflowRunner because of JSON string decoding error

["Sam Rohde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733675#comment-16733675 ] 

Sam Rohde commented on BEAM-5788:
---------------------------------

[~markflyhigh] ping

Can you please take a look?

> wordcount_fnapi_it failed on TestDataflowRunner because of JSON string decoding error
> -------------------------------------------------------------------------------------
>
>                 Key: BEAM-5788
>                 URL: https://issues.apache.org/jira/browse/BEAM-5788
>             Project: Beam
>          Issue Type: Sub-task
>          Components: test-failures
>            Reporter: Mark Liu
>            Assignee: Mark Liu
>            Priority: Major
>          Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Similar to BEAM-5785, wordcount_fnapi_it failed on Python 3 when running with TestDataflowRunner. Got TypeError: the JSON object must be str, not 'bytes'. This error cause infinite retry before job could submitted to service.
> More details about my env and test:
> Python version: 3.5.3
> Test: apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it
> Command:
> {code}
> python setup.py nosetests \
>   --tests apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_fnapi_it  \
>   --nocapture \
>   --nologcapture \
>   --test-pipeline-options=" \                             
>         --runner=TestDataflowRunner \
>         --project=<my_project> \
>         --staging_location=<my_staging> \
>         --temp_location=<my_temp> \
>         --output=<my_output> \                                                                      
>         --sdk_location=.../beam/sdks/python/dist/apache-beam-2.9.0.dev0.tar.gz \
>         --num_workers=1"
> {code}
> Stacktrace:
> {code}
> WARNING:root:Retry with exponential backoff: waiting for 7.661876827680761 seconds before retrying exists because we caught exception: TypeError: the JSON object must be str, not 'bytes'
>  Traceback for above exception (most recent call last):
>   File ".../beam/sdks/python/apache_beam/utils/retry.py", line 184, in wrapper
>     return fun(*args, **kwargs)
>   File ".../beam/sdks/python/apache_beam/io/gcp/gcsio.py", line 375, in exists
>     self.client.objects.Get(request)  # metadata
>   File ".../beam/sdks/python/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py", line 955, in Get
>     download=download)
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 722, in _RunMethod
>     return self.ProcessHttpResponse(method_config, http_response, request)
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 728, in ProcessHttpResponse
>     self.__ProcessHttpResponse(method_config, http_response, request))
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 611, in __ProcessHttpResponse
>     response_type, http_response.content)
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 442, in DeserializeMessage
>     message = encoding.JsonToMessage(response_type, data)
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py", line 104, in JsonToMessage
>     return _ProtoJsonApiTools.Get().decode_message(message_type, message)
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/py/encoding.py", line 290, in decode_message
>     message_type, result)
>   File ".../tmp/virtualenvs/py3-env/lib/python3.5/site-packages/apitools/base/protorpclite/protojson.py", line 210, in decode_message
>     dictionary = json.loads(encoded_message)
>   File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
>     s.__class__.__name__))
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)