You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "scwhittle (via GitHub)" <gi...@apache.org> on 2023/11/01 14:45:48 UTC
[I] [Bug]: Change reflection-based schema generation to generate deterministic schemas [beam]
scwhittle opened a new issue, #29245:
URL: https://github.com/apache/beam/issues/29245
### What happened?
Currently schema fields are assigned ordering based upon ordering of ReflectUtils.getMethods and ReflectUtils.getFields.
ReflectUtils.getMethods ordering is based upon Class.getDeclaredMethods which has no specified ordering and may differ across JVMs. It appears that it must be somewhat consistent ordering in our test environment because changing the ordering to sort breaks tests relying on the current ordering.
ReflectUtils.getFields returns ordering based upon HashMap iteration over field names. There are tests that depend on this hash ordering as well (which might not be consistent if hashes are randomized as in some JVMs).
We should modify these methods to return a consistent ordering so that generated schemas for unmodified classes are consistent and not depending on unspecified ordering.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Change reflection-based schema generation to generate deterministic schemas [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #29245:
URL: https://github.com/apache/beam/issues/29245#issuecomment-4799689127
This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Change reflection-based schema generation to generate deterministic schemas [beam]
Posted by "scwhittle (via GitHub)" <gi...@apache.org>.
scwhittle commented on issue #29245:
URL: https://github.com/apache/beam/issues/29245#issuecomment-1789086448
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Change reflection-based schema generation to generate deterministic schemas [beam]
Posted by "scwhittle (via GitHub)" <gi...@apache.org>.
scwhittle commented on issue #29245:
URL: https://github.com/apache/beam/issues/29245#issuecomment-1916859864
Another solution to the problem would be to determine where the non-deterministic ordering used between instances causes issues. My suspicion is that they are embedded in the toRow/fromRow functions. A different approach to sorting fields would be to randomize the indices and see what breaks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Change reflection-based schema generation to generate deterministic schemas [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #29245: [Bug]: Change reflection-based schema generation to generate deterministic schemas
URL: https://github.com/apache/beam/issues/29245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Change reflection-based schema generation to generate deterministic schemas [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #29245:
URL: https://github.com/apache/beam/issues/29245#issuecomment-4534719920
This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org