Re: Count on non-global windows

[Kenneth Knowles <kl...@google.com>]

In fact, there is a framework in place for giving a better error message
that is just not used. Filed https://issues.apache.org/jira/browse/BEAM-3746
(maybe took longer to file than to fix, but I'm just not at an IDE right
now).

On Mon, Feb 26, 2018 at 8:26 AM, Kenneth Knowles <kl...@google.com> wrote:

> Hi Vilhelm,
>
> Sorry, this is not a good experience. You analysis is spot on, and the
> workaround you suggested is what we have done. It is a known issue that
> there is a tension between all the helper methods of Combine and the nice
> wrappers like Count. So for now it is just in the fine print of the
> documentation: https://beam.apache.org/documentation/sdks/
> javadoc/2.3.0/org/apache/beam/sdk/transforms/Count.html#globally--
>
> "Note: if the input collection uses a windowing strategy other than
> GlobalWindows, use Combine.globally(Count.<T>combineFn()).withoutDefaults()
> instead."
>
> Kenn
>
> On Mon, Feb 26, 2018 at 2:47 AM, Vilhelm von Ehrenheim <
> vonehrenheim@gmail.com> wrote:
>
>> Hi!
>> I am trying to use the `Count` Combine transform on Fixed windows and get
>> the following error:
>>
>> ```
>> java.lang.IllegalStateException: Default values are not supported in
>> Combine.globally() if the output PCollection is not windowed by
>> GlobalWindows. Instead, use Combine.globally().withoutDefaults() to
>> output an empty PCollection if the input PCollection is empty, or
>> Combine.globally().asSingletonView() to get the default output of the
>> CombineFn if the input PCollection is empty.
>> ```
>>
>> I get that you need to set this on combiners for non-global windows but
>> it is not possible to set for the `Count.globally()` transform. I can
>> implement this using my own CompbineFn or `Sum` transform instead of course
>> but still thought it was a bit strange.
>>
>> Is this a bug or why is this happening?
>>
>> Regards,
>> Vilhelm von Ehrenheim
>>
>
>