You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Luke Cwik <lc...@google.com> on 2021/08/09 21:51:57 UTC
Re: Unable to use windowing transform on Dataflow with Go SDK (BEAM-12636)
+Robert Burke <re...@google.com>
On Fri, Jul 23, 2021 at 1:50 AM Hannes Gustafsson <
hannes@hoxtonanalytics.com> wrote:
> Hi,
>
> I've now run into https://issues.apache.org/jira/browse/BEAM-12636 and
> I'm unable to use windowing in my Go SDK pipeline on Dataflow.
>
>
> If anyone has input on what might be the cause that'd be very helpful as
> I'm looking into fixing it but do not have much experience with code base.
>
>
>
> Thanks,
>
Re: Unable to use windowing transform on Dataflow with Go SDK
(BEAM-12636)
Posted by Hannes Gustafsson <ha...@hoxtonanalytics.com>.
In case it is helpful, as noted in the linked issue this happens on 2.31.0 but I've since been able to build against a local clone of master and submit my job successfully. I've just now rebuilt against 2fd98755f2 and again successfully submitted a job that uses fixed windows.
I've not been able yet to debug this further and am otherwise holding out for the looming triggering support in https://github.com/apache/beam/pull/15239 to replace my current window usage.
________________________________
From: Luke Cwik <lc...@google.com>
Sent: 09 August 2021 22:51
To: user <us...@beam.apache.org>; Robert Burke <re...@google.com>
Subject: Re: Unable to use windowing transform on Dataflow with Go SDK (BEAM-12636)
+Robert Burke<ma...@google.com>
On Fri, Jul 23, 2021 at 1:50 AM Hannes Gustafsson <ha...@hoxtonanalytics.com>> wrote:
Hi,
I've now run into https://issues.apache.org/jira/browse/BEAM-12636 and I'm unable to use windowing in my Go SDK pipeline on Dataflow.
If anyone has input on what might be the cause that'd be very helpful as I'm looking into fixing it but do not have much experience with code base.
Thanks,
Re: Unable to use windowing transform on Dataflow with Go SDK (BEAM-12636)
Posted by Robert Burke <re...@google.com>.
I've replied on the JIRA issue but to repeat myself here:
The windowing example was broken for a while, up to and including version
2.31.0. Windowing in the Go SDK should be fixed in 2.32.0. However, the Go
SDK remains experimental for that version as well.
Currently we're working towards making 2.33.0 be the first non-experimental
version of the Go SDK, with proper Go Modules support among other things.
When there's news on this change of status, there will be a blogpost on the
beam blog with a rundown on the state of the SDK.
Finally, per usual: Dataflow doesn't officially support the Go SDK at this
time. Exiting Experimental is a necessary, but not sufficient condition for
Dataflow to support the SDK.
Cheers,
Robert Burke
Beam Go Busybody
On Mon, Aug 9, 2021 at 2:52 PM Luke Cwik <lc...@google.com> wrote:
> +Robert Burke <re...@google.com>
>
> On Fri, Jul 23, 2021 at 1:50 AM Hannes Gustafsson <
> hannes@hoxtonanalytics.com> wrote:
>
>> Hi,
>>
>> I've now run into https://issues.apache.org/jira/browse/BEAM-12636 and
>> I'm unable to use windowing in my Go SDK pipeline on Dataflow.
>>
>>
>> If anyone has input on what might be the cause that'd be very helpful as
>> I'm looking into fixing it but do not have much experience with code base.
>>
>>
>>
>> Thanks,
>>
>