You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/08/16 21:30:08 UTC

[GitHub] [beam] chamikaramj opened a new issue, #22748: [Bug]: ValidateRunnerXlangTest$SingleInputOutputTest.test failing for various x-lang test suites

chamikaramj opened a new issue, #22748:
URL: https://github.com/apache/beam/issues/22748

   ### What happened?
   
   For example, 
   
   https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/5756/
   https://ci-beam.apache.org/job/beam_PostCommit_XVR_JavaUsingPython_Dataflow/787/
   https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/3008/
   
   Stacktrace
   java.lang.RuntimeException: expansion service error: Traceback (most recent call last):
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/runners/portability/expansion_service.py", line 65, in Expand
       ptransform.PTransform.from_runner_api(request.transform, context))
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 768, in from_runner_api
       context)
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/runners/portability/expansion_service_test.py", line 113, in from_runner_api_parameter
       return PrefixTransform(parse_string_payload(payload)['data'])
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/runners/portability/expansion_service_test.py", line 349, in parse_string_payload
       return RowCoder(payload.schema).decode(payload.payload)._asdict()
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/coders/row_coder.py", line 64, in __init__
       self._type_hint = named_tuple_from_schema(self.schema)
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/typehints/schemas.py", line 473, in named_tuple_from_schema
       schema_registry=schema_registry).named_tuple_from_schema(schema)
     File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_Direct/src/build/gradleenv/1922375555/lib/python3.7/site-packages/apache_beam/typehints/schemas.py", line 432, in named_tuple_from_schema
       assert schema.id
   AssertionError
   
   ### Issue Priority
   
   Priority: 1
   
   ### Issue Component
   
   Component: test-failures


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] yeandy commented on issue #22748: [Bug]: ValidateRunnerXlangTest$SingleInputOutputTest.test failing for various x-lang test suites

Posted by GitBox <gi...@apache.org>.
yeandy commented on issue #22748:
URL: https://github.com/apache/beam/issues/22748#issuecomment-1217213543

   I believe we can remove the assertion. In https://github.com/apache/beam/pull/22679, most of the code around it was simply refactored with non change, except for the addition of that particular line.
   
   I'm guessing the value during these tests is either `''` or `None`, which doesn't actually break the tests. (Is this intended though?)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on issue #22748: [Bug]: ValidateRunnerXlangTest$SingleInputOutputTest.test failing for various x-lang test suites

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on issue #22748:
URL: https://github.com/apache/beam/issues/22748#issuecomment-1223096162

   Thanks @chamikaramj and @yeandy for addressing this while I was out. IIUC the action here now is to make sure that the Java SDK always generates an ID for schemas (then potentially add the assertion back in Python).
   
   Lowering to P2 since this is now feature work.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] chamikaramj commented on issue #22748: [Bug]: ValidateRunnerXlangTest$SingleInputOutputTest.test failing for various x-lang test suites

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on issue #22748:
URL: https://github.com/apache/beam/issues/22748#issuecomment-1217287518

   ID is probably set to empty string which is done here: https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaTranslation.java#L83
   
   According to the proto, the ID has to be set to a UUID: https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/schema.proto#L40
   
   So the actual fix might have to be done to the SchemaTranslation class (and any other locations where we do not set an ID when generating the Schema proto).
   
   I'll merge https://github.com/apache/beam/pull/22750 though to fix the tests.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] chamikaramj commented on issue #22748: [Bug]: ValidateRunnerXlangTest$SingleInputOutputTest.test failing for various x-lang test suites

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on issue #22748:
URL: https://github.com/apache/beam/issues/22748#issuecomment-1217189466

   @yeandy will you be able to check since @TheNeuralBit is out ? Can we just remove the assertion ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] chamikaramj commented on issue #22748: [Bug]: ValidateRunnerXlangTest$SingleInputOutputTest.test failing for various x-lang test suites

Posted by GitBox <gi...@apache.org>.
chamikaramj commented on issue #22748:
URL: https://github.com/apache/beam/issues/22748#issuecomment-1217186622

   Potential culprit: https://github.com/apache/beam/pull/22679
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org