You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "rulle-io (via GitHub)" <gi...@apache.org> on 2023/06/15 07:38:30 UTC
[GitHub] [beam] rulle-io opened a new issue, #27138: [Bug]: FirestoreIO writes fails: timer is set to max time (294247-01-09T04:00:54.775Z)
rulle-io opened a new issue, #27138:
URL: https://github.com/apache/beam/issues/27138
### What happened?
```
Caused by: java.lang.IllegalArgumentException: Cannot output with timestamp 2023-06-15T07:19:21.297Z. Output timestamps must be no earlier than the timestamp of the current input or timer (294247-01-09T04:00:54.775Z) minus the allowed skew (0 milliseconds) and no later than 294247-01-10T04:00:54.775Z. See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew.
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.checkTimestamp(SimpleDoFnRunner.java:259)
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1300(SimpleDoFnRunner.java:85)
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.outputWithTimestamp(SimpleDoFnRunner.java:416)
org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn$BaseBatchWriteFn$ProcessContextAdapter.output(FirestoreV1WriteFn.java:557)
org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn$BatchWriteFnWithSummary.handleWriteSummary(FirestoreV1WriteFn.java:96)
org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn$BaseBatchWriteFn.doFlush(FirestoreV1WriteFn.java:432)
org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn$BaseBatchWriteFn.flushBatch(FirestoreV1WriteFn.java:304)
org.apache.beam.sdk.io.gcp.firestore.FirestoreV1WriteFn$BaseBatchWriteFn.processElement(FirestoreV1WriteFn.java:256)
```
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z) [beam]
Posted by "djaneluz (via GitHub)" <gi...@apache.org>.
djaneluz commented on issue #27138:
URL: https://github.com/apache/beam/issues/27138#issuecomment-1824899543
I'm facing the same error, my batch pipeline does as follows.
```
pipeline
.apply("GetBigQueryInfo", BigQueryIO.readTableRows()
.fromQuery(getDeltaQuery(projectId, firestoreStateDataset))
.usingStandardSql())
.apply("TableRowToWrite", ParDo.of(new TableRowToWriteFn(projectId, databaseId)))
.apply(GroupByKey.create())
.apply(Values.create())
.apply(Flatten.iterables())
.apply("WriteToFirestore", FirestoreIO.v1().write()
.batchWrite()
.withRpcQosOptions(rpcQosOptions)
.build());
```
and the `WriteToFirestore` step fails wit the exception.
I'm using BEAM 2.52.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z) [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles closed issue #27138: [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z)
URL: https://github.com/apache/beam/issues/27138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AndreiArion commented on issue #27138: [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z)
Posted by "AndreiArion (via GitHub)" <gi...@apache.org>.
AndreiArion commented on issue #27138:
URL: https://github.com/apache/beam/issues/27138#issuecomment-1594831517
Hello,
We are experience the same issue with the latest Apache Beam version (Java 2.48.0)
All our writes are blocked. Is there any workaround ?
Thanks,
Andrei
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z) [beam]
Posted by "stankiewicz (via GitHub)" <gi...@apache.org>.
stankiewicz commented on issue #27138:
URL: https://github.com/apache/beam/issues/27138#issuecomment-3306417740
.take_issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z) [beam]
Posted by "djaneluz (via GitHub)" <gi...@apache.org>.
djaneluz commented on issue #27138:
URL: https://github.com/apache/beam/issues/27138#issuecomment-1824931734
**UPDATE**
I followed the recommendation from this article https://medium.com/@erik.b.ferm/combining-bounded-pcollections-with-custom-timestamps-in-apache-beam-b0fd911b2ac and it worked.
My pipeline now is like this:
```
pipeline
.apply("GetBigQueryInfo", BigQueryIO.readTableRows()
.fromQuery(getDeltaQuery(projectId, firestoreStateDataset))
.usingStandardSql())
.apply("TableRowToWrite", ParDo.of(new TableRowToWriteFn(projectId, databaseId)))
.apply("SetTimeStampCombiner", Window.<KV<String, Write>>into(new GlobalWindows())
.withTimestampCombiner(TimestampCombiner.EARLIEST))
.apply(GroupByKey.create())
.apply(Values.create())
.apply(Flatten.iterables())
.apply("WriteToFirestore", FirestoreIO.v1().write()
.batchWrite()
.withRpcQosOptions(rpcQosOptions)
.build());
```
So the solution was to explicitly add a `GlobalWindow` using `.withTimestampCombiner(TimestampCombiner.EARLIEST)` before the `GroupByKey`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AndreiArion commented on issue #27138: [Bug]: FirestoreIO writes fail: timer is set to max time (294247-01-09T04:00:54.775Z)
Posted by "AndreiArion (via GitHub)" <gi...@apache.org>.
AndreiArion commented on issue #27138:
URL: https://github.com/apache/beam/issues/27138#issuecomment-1608604933
Our pipeline reads data from a Firestore collection, do some transformations and and write it back to Firestore.
The erreur occurs during the write to firestore phase but the cause seems to be that FirestoreV1 source generates bad timestamps for some elements.
A workaround that worked in our context (batch processing) was to change the timestamp for all elements :
`
myCollection.apply("Pardo to map to Write objects , ParDo.of(
new MyWriteConverterDoFn()
))
// Firestore Sink provides a bad timestamp(MAX_LONG), so we need to fix it
.apply("Change timestamp of all elements",
WithTimestamps.of((SerializableFunction<Write, org.joda.time.Instant>) input -> {
assert input != null;
return org.joda.time.Instant.now();
}).withAllowedTimestampSkew(new org.joda.time.Duration(Long.MAX_VALUE)))
.apply("Write PiiUsers to Firestore",
FirestoreIO.v1().write().batchWrite().withRpcQosOptions(rpcQosOptions).build())
`
Please see all the implications in the documentation of withAllowedTimestampSkew method as it might not work in your setting !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org