You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "ujjwalrajanand (via GitHub)" <gi...@apache.org> on 2024/10/18 18:18:37 UTC
[I] [Failing Test]: Processing failed records [beam]
ujjwalrajanand opened a new issue, #32866:
URL: https://github.com/apache/beam/issues/32866
### What happened?
https://github.com/vaishnavipandey-vp/Apache-Beam-YAML/blob/c467c59f5f13a00514e90be1cfd505982577524b/Apache-Beam-YAML/Pipelines/exceptionHandlingMultiple.yaml#L19C1-L28C35
Not able to process the failed records obtained from the previous transform as it is generated in a JSON format. We are required to extract the element values of the failed records from the JSON file in order to process it further.
### Issue Failure
Failure: Test is continually failing
### Issue Priority
Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [X] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Difficulty reprocessing failed records [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on issue #32866:
URL: https://github.com/apache/beam/issues/32866#issuecomment-2423279405
I created https://github.com/apache/beam/pull/32874 to make the documentation a bit clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Difficulty reprocessing failed records [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb closed issue #32866: Difficulty reprocessing failed records
URL: https://github.com/apache/beam/issues/32866
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Difficulty reprocessing failed records [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on issue #32866:
URL: https://github.com/apache/beam/issues/32866#issuecomment-2423331018
I started a discussion at https://lists.apache.org/thread/jhgsp1rw7l1xr6knpjl9mhvnjwjf3zwf about how to possibly make this easier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Difficulty reprocessing failed records [beam]
Posted by "robertwb (via GitHub)" <gi...@apache.org>.
robertwb commented on issue #32866:
URL: https://github.com/apache/beam/issues/32866#issuecomment-2524603074
This is now resolved with the [StripErrorMetadata](https://beam.apache.org/releases/yamldoc/current/#striperrormetadata) transform, see also https://beam.apache.org/documentation/sdks/yaml-errors/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org