You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/07 18:12:13 UTC
[GitHub] [beam] reuvenlax opened a new issue, #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
reuvenlax opened a new issue, #22188:
URL: https://github.com/apache/beam/issues/22188
### What happened?
The idle timer by default tries to output to an invalid timestamp.
### Issue Priority
Priority: 1
### Issue Component
Component: io-java-gcp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] anish97IND commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
anish97IND commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1186848037
> We expect this to be fixed in 2.40.1 - unfortunately we did not find this bug before 2.40.0 was released.
> […](#)
> On Fri, Jul 15, 2022 at 2:47 AM Anish Kumar Sarangi < ***@***.***> wrote: Hi , We are facing this issue with Apache Beam 2.40.0 and also on 2.39, is it something to be fixed in next release? Adding Logs here for reference-: java.lang.IllegalStateException: TimestampCombiner moved element from 294247-01-10T04:00:54.775Z (TIMESTAMP_MAX_VALUE) to earlier time 294247-01-09T04:00:54.775Z (end of global window) for window ***@***.*** -: 2.39.0. And on 2.40.0 Following error is experienced, is it something which will also get fixed?-: LOG-: java.lang.IllegalArgumentException: Cannot output with timestamp 294247-01-10T04:00:54.776Z. Output timestamps must be no earlier than the timestamp of the current input or timer (294247-01-10T04:00:54.776Z) minus the allowed skew (0 milliseconds) and no later than 294247-01-10T04:00:54.775Z. See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew. — Reply to this email directly, view it on GitHub <[#22188 (comment)
](https://github.com/apache/beam/issues/22188#issuecomment-1185374497)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFAYJVIMZ7SP77T4LX5FFDLVUEXSNANCNFSM526KATWA> . You are receiving this because you were mentioned.Message ID: ***@***.***>
Thanks for the update! Much appreciated! By any chance we can get a view of how soon that would be happening?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jccarles commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by "jccarles (via GitHub)" <gi...@apache.org>.
jccarles commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1521828027
Hello ! Is this fixed in the latest releases ? I am still facing this issue I believe. I am using python SDK version 2.46 (I'm running on flink).
```
Caused by: java.lang.IllegalStateException: TimestampCombiner moved element from 294247-01-10T04:00:54.775Z (TIMESTAMP_MAX_VALUE) to earlier time 294247-01-09T04:00:54.775Z (end of global window) for window org.apache.beam.sdk.transforms.windowing.GlobalWindow@1bc6bee6
```
I'm facing this issue trying to run StatisticGen TFX Component if it helps...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jccarles commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by "jccarles (via GitHub)" <gi...@apache.org>.
jccarles commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1541565964
Sure, thank you both !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1540772797
Good point. @jccarles - you can file a separate issue and provide a minimal repro to if you are still affected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] reuvenlax commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
reuvenlax commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1201833561
@kileys I think we need to get this into the release, yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] anish97IND commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
anish97IND commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1185374497
Hi , We are facing this issue with Apache Beam 2.40.0 and also on 2.39, is it something to be fixed in next release? Adding Logs here for reference-: java.lang.IllegalStateException: TimestampCombiner moved element from 294247-01-10T04:00:54.775Z (TIMESTAMP_MAX_VALUE) to earlier time 294247-01-09T04:00:54.775Z (end of global window) for window org.apache.beam.sdk.transforms.windowing.GlobalWindow@1d34804b -: 2.39.0.
And on 2.40.0 Following error is experienced, is it something which will also get fixed?-:
LOG-: java.lang.IllegalArgumentException: Cannot output with timestamp 294247-01-10T04:00:54.776Z. Output timestamps must be no earlier than the timestamp of the current input or timer (294247-01-10T04:00:54.776Z) minus the allowed skew (0 milliseconds) and no later than 294247-01-10T04:00:54.775Z. See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp [beam]
Posted by "keerthy17190 (via GitHub)" <gi...@apache.org>.
keerthy17190 commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1854460754
We are facing the below error in sdk version 2.48.0. Will this issue be resolved by upgrading the
sdk version
org.apache.beam.sdk.util.UserCodeException: java.lang.IllegalArgumentException: Cannot output with timestamp 2023-12-13T06:48:59.792Z. Output timestamps must be no earlier than the timestamp of the current input or timer (2023-12-13T06:49:00.579Z) minus the allowed skew (0 milliseconds) and no later than 294247-01-10T04:00:54.775Z. See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] reuvenlax commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
reuvenlax commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1182124723
@johnjcasey This should now be fixed. This usually happens when the sink goes idle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1234632617
What SDK versions are affected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] johnjcasey commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
johnjcasey commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1180866994
@reuvenlax Do you know under what circumstances this happens, or a way to repro this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kileys commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
kileys commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1201534366
@johnjcasey @reuvenlax Are we blocking release on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] johnjcasey commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
johnjcasey commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1182320559
what was the fix jira? just want to confirm before we close this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] reuvenlax commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
reuvenlax commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1182335293
https://github.com/apache/beam/pull/22119
On Tue, Jul 12, 2022 at 12:14 PM johnjcasey ***@***.***>
wrote:
> what was the fix jira? just want to confirm before we close this
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/issues/22188#issuecomment-1182320559>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFAYJVN3BDSEKCIP7XDDSXDVTW7ZTANCNFSM526KATWA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kileys closed issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
kileys closed issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
URL: https://github.com/apache/beam/issues/22188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] reuvenlax commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1538746728
The PR fixed the Java sink. I believe that Python is using a completely
separate sink implementation.
On Tue, Apr 25, 2023 at 7:13 AM tvalentyn ***@***.***> wrote:
> According to PR metadata this is expected to have been fixed as of 2.41.0.
> @reuvenlax <https://github.com/reuvenlax> might now if there are any
> other similar issues in newer releases.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/issues/22188#issuecomment-1521864559>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFAYJVPATBKN5BUCZGHZ263XC7LXJANCNFSM526KATWA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1521864559
According to PR metadata this is expected to have been fixed as of 2.41.0. @reuvenlax might now if there are any other similar issues in newer releases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] reuvenlax commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
reuvenlax commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1186396794
We expect this to be fixed in 2.40.1 - unfortunately we did not find this
bug before 2.40.0 was released.
On Fri, Jul 15, 2022 at 2:47 AM Anish Kumar Sarangi <
***@***.***> wrote:
> Hi , We are facing this issue with Apache Beam 2.40.0 and also on 2.39, is
> it something to be fixed in next release? Adding Logs here for reference-:
> java.lang.IllegalStateException: TimestampCombiner moved element from
> 294247-01-10T04:00:54.775Z (TIMESTAMP_MAX_VALUE) to earlier time
> 294247-01-09T04:00:54.775Z (end of global window) for window
> ***@***.*** -: 2.39.0.
> And on 2.40.0 Following error is experienced, is it something which will
> also get fixed?-:
> LOG-: java.lang.IllegalArgumentException: Cannot output with timestamp
> 294247-01-10T04:00:54.776Z. Output timestamps must be no earlier than the
> timestamp of the current input or timer (294247-01-10T04:00:54.776Z) minus
> the allowed skew (0 milliseconds) and no later than
> 294247-01-10T04:00:54.775Z. See the DoFn#getAllowedTimestampSkew() Javadoc
> for details on changing the allowed skew.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/issues/22188#issuecomment-1185374497>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFAYJVIMZ7SP77T4LX5FFDLVUEXSNANCNFSM526KATWA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kileys commented on issue #22188: BigQuery Storage API sink sometimes gets stuck outputting to an invalid timestamp
Posted by GitBox <gi...@apache.org>.
kileys commented on issue #22188:
URL: https://github.com/apache/beam/issues/22188#issuecomment-1197065076
Is this fixed by https://github.com/apache/beam/pull/22119?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org