You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/08/18 11:18:16 UTC
[GitHub] [beam] sheepdreamofandroids opened a new issue, #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
sheepdreamofandroids opened a new issue, #22773:
URL: https://github.com/apache/beam/issues/22773
### What happened?
We have a relatively simple pipeline reading from Pubsub and writing to ElasticsearchIO running in the DataflowRunner.
This fails in beam versions where ElasticsearchIO.BulkIO.BulkIOBaseFn.ProcessContextAdapter#output uses outputWithTimestamp() and succeeds where it uses output(). The git history shows that the implementation bounces back and forth between both implementations. ATM we're stuck with version 2.38.0 since that is the most recent that works.
I thought this might be related to the pubsub clients using outputWithTimestamp with the original publication time. But an inbetween ParDo that uses outputWithTimestamp with the current time did not help.
The error we get:
```
java.lang.IllegalArgumentException: Cannot output with timestamp 2022-08-16T11:03:57.086Z. Output timestamps must be no earlier than the timestamp of the current input or timer (2022-08-16T11:33:50.762Z) minus the allowed skew (0 milliseconds) and no later than 294247-01-10T04:00:54.775Z. See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew.
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.checkTimestamp(SimpleDoFnRunner.java:259)
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1300(SimpleDoFnRunner.java:85)
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.outputWithTimestamp(SimpleDoFnRunner.java:416)
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$BulkIO$BulkIOBaseFn$ProcessContextAdapter.output(ElasticsearchIO.java:2416)
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$BulkIO$BulkIOBaseFn.flushAndOutputResults(ElasticsearchIO.java:2436)
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$BulkIO$BulkIOBaseFn.addAndMaybeFlush(ElasticsearchIO.java:2448)
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO$BulkIO$BulkIOStatefulFn.processElement(ElasticsearchIO.java:2349)
```
I'm not sure about the P1 prio.
### Issue Priority
Priority: 1
### Issue Component
Component: io-java-elasticsearch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sheepdreamofandroids commented on issue #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
Posted by GitBox <gi...@apache.org>.
sheepdreamofandroids commented on issue #22773:
URL: https://github.com/apache/beam/issues/22773#issuecomment-1226928552
It's running in test now and it looks good so far. 2.40.0 failed immediately so I'm very confident that this is solved.
Great work!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sheepdreamofandroids commented on issue #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
Posted by GitBox <gi...@apache.org>.
sheepdreamofandroids commented on issue #22773:
URL: https://github.com/apache/beam/issues/22773#issuecomment-1219658940
Wow, that is fast! Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] egalpin commented on issue #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
Posted by GitBox <gi...@apache.org>.
egalpin commented on issue #22773:
URL: https://github.com/apache/beam/issues/22773#issuecomment-1226315037
@sheepdreamofandroids 2.41.0 is released now, could you please try upgrading to 2.41.0 and confirm that the issue is solved from your perspective? I've just updated a running job sinking to ES from v2.38.0 to v2.41.0 and this issue is no longer present (whereas it was in 2.39.0/2.40.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] egalpin commented on issue #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
Posted by GitBox <gi...@apache.org>.
egalpin commented on issue #22773:
URL: https://github.com/apache/beam/issues/22773#issuecomment-1219473185
Thanks for raising this as it is a major issue. This issue is fixed by https://github.com/apache/beam/pull/22183.
When 2.41.0 is released (which is likely to happen within a week or two), this PR will be included and this issue should be fixed for pipeline developers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sheepdreamofandroids closed issue #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
Posted by GitBox <gi...@apache.org>.
sheepdreamofandroids closed issue #22773: [Bug]: ElasticsearchIO.Write fails when calling outputWithTimestamp()
URL: https://github.com/apache/beam/issues/22773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org