How to create a custom trigger?

[Shen Li <cs...@gmail.com>]

Hi,

I created a custom trigger class (XYZ) by extending the OnceTrigger. During
execution, I got this error "Cannot translate trigger class XYZ to a
runner-API proto." It seems that the Triggers.ProtoConverter class needs to
declare a convertSpecific method for my trigger XYZ. How can I use my
custom trigger without modifying Beam's Triggers class?

Thanks,

Shen

Re: How to create a custom trigger?

[Lukasz Cwik <lc...@google.com.INVALID>]

Just to circle back, making the Runner TestStream compatible would allow
for exercising existing tests for trigger functionality. The DirectRunner
is the only one which currently does this but we hope all Runners will
follow so that each Runner has a suite of tests they can rely on to verify
triggering semantics/behavior.

On Wed, Jun 7, 2017 at 11:39 AM, Shen Li <cs...@gmail.com> wrote:

> Hi Kenn,
>
> Thanks for the explanation. Yes, I will test the runner directly.
>
> Shen
>
> On Wed, Jun 7, 2017 at 12:35 PM, Kenneth Knowles <kl...@google.com.invalid>
> wrote:
>
> > Hi Shen,
> >
> > There are four points (at least) to why we don't currently have custom
> > triggers:
> >
> > 1. The existing trigger state machine implementations use low-level and
> > difficult APIs that are not available to users.
> > 2. If we make triggers a UDF then we need to cross the Fn API. It makes a
> > lot more portability work in terms of engineering and also is really the
> > center of a tight loop. Instead, runners can very efficiently execute
> them
> > directly.
> > 3. There are many state machines that are not valid triggers.
> > 4. Some triggers can be more efficiently executed directly from their
> > syntax than from a state machine.
> >
> > Eventually, we may add some kind of triggering UDF. But we may not,
> because
> > if you want something like a custom trigger for some use case, then you
> can
> > do it with state & timers in ParDo.
> >
> > But it sounds like you need it for some quite specific testing. Since
> > "watermark advance event" is a runner-specific concept not part of the
> > model (a runner doesn't have to actually reify such events) maybe you
> just
> > want to test your runner details directly.
> >
> > Kenn
> >
> > On Wed, Jun 7, 2017 at 10:22 AM, Shen Li <cs...@gmail.com> wrote:
> >
> > > Hi Lukasz,
> > >
> > > Thanks again for the suggestion. Is there any reason for not allowing
> > users
> > > create custom triggers?
> > >
> > > Shen
> > >
> > > On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <lcwik@google.com.invalid
> >
> > > wrote:
> > >
> > > > You should really take a look at TestStream and have runners
> integrate
> > > with
> > > > it instead.
> > > >
> > > > There are already several tests which validate TestStream compatible
> > > > runners to make sure their trigger evaluations are correct.
> > > >
> > > > On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com>
> wrote:
> > > >
> > > > > Hi Lukasz,
> > > > >
> > > > > Thanks for the suggestion. I am trying to test how the runner
> > generates
> > > > > watermarks. So I would like to have the trigger to fire on every
> > > > watermark
> > > > > advancing event.
> > > > >
> > > > > Shen
> > > > >
> > > > > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik
> > <lcwik@google.com.invalid
> > > >
> > > > > wrote:
> > > > >
> > > > > > Look into the AfterPane#elementCountAtLeast trigger as it seems
> to
> > be
> > > > the
> > > > > > closest to your description. It fires as soon as any data is
> > > available.
> > > > > >
> > > > > > Are you sure you don't want some kind of watermark based trigger
> > with
> > > > > just
> > > > > > with a small interval size?
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > Hi Lukasz,
> > > > > > >
> > > > > > > Thanks for your response. Is it possible to implement the
> > following
> > > > > logic
> > > > > > > using existing triggers: always fire the trigger on a
> > GlobalWindow
> > > > > > whenever
> > > > > > > watermark advances?
> > > > > > >
> > > > > > > Shen
> > > > > > >
> > > > > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik
> > > > <lcwik@google.com.invalid
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Users are unable to create custom trigger implementations. If
> > you
> > > > > tell
> > > > > > us
> > > > > > > > what you want your trigger to do, we may be able to suggest
> an
> > > > > > > alternative.
> > > > > > > >
> > > > > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs.shenli@gmail.com
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I created a custom trigger class (XYZ) by extending the
> > > > > OnceTrigger.
> > > > > > > > During
> > > > > > > > > execution, I got this error "Cannot translate trigger class
> > XYZ
> > > > to
> > > > > a
> > > > > > > > > runner-API proto." It seems that the
> Triggers.ProtoConverter
> > > > class
> > > > > > > needs
> > > > > > > > to
> > > > > > > > > declare a convertSpecific method for my trigger XYZ. How
> can
> > I
> > > > use
> > > > > my
> > > > > > > > > custom trigger without modifying Beam's Triggers class?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Shen
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Shen Li <cs...@gmail.com>]

Hi Kenn,

Thanks for the explanation. Yes, I will test the runner directly.

Shen

On Wed, Jun 7, 2017 at 12:35 PM, Kenneth Knowles <kl...@google.com.invalid>
wrote:

> Hi Shen,
>
> There are four points (at least) to why we don't currently have custom
> triggers:
>
> 1. The existing trigger state machine implementations use low-level and
> difficult APIs that are not available to users.
> 2. If we make triggers a UDF then we need to cross the Fn API. It makes a
> lot more portability work in terms of engineering and also is really the
> center of a tight loop. Instead, runners can very efficiently execute them
> directly.
> 3. There are many state machines that are not valid triggers.
> 4. Some triggers can be more efficiently executed directly from their
> syntax than from a state machine.
>
> Eventually, we may add some kind of triggering UDF. But we may not, because
> if you want something like a custom trigger for some use case, then you can
> do it with state & timers in ParDo.
>
> But it sounds like you need it for some quite specific testing. Since
> "watermark advance event" is a runner-specific concept not part of the
> model (a runner doesn't have to actually reify such events) maybe you just
> want to test your runner details directly.
>
> Kenn
>
> On Wed, Jun 7, 2017 at 10:22 AM, Shen Li <cs...@gmail.com> wrote:
>
> > Hi Lukasz,
> >
> > Thanks again for the suggestion. Is there any reason for not allowing
> users
> > create custom triggers?
> >
> > Shen
> >
> > On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <lc...@google.com.invalid>
> > wrote:
> >
> > > You should really take a look at TestStream and have runners integrate
> > with
> > > it instead.
> > >
> > > There are already several tests which validate TestStream compatible
> > > runners to make sure their trigger evaluations are correct.
> > >
> > > On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com> wrote:
> > >
> > > > Hi Lukasz,
> > > >
> > > > Thanks for the suggestion. I am trying to test how the runner
> generates
> > > > watermarks. So I would like to have the trigger to fire on every
> > > watermark
> > > > advancing event.
> > > >
> > > > Shen
> > > >
> > > > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik
> <lcwik@google.com.invalid
> > >
> > > > wrote:
> > > >
> > > > > Look into the AfterPane#elementCountAtLeast trigger as it seems to
> be
> > > the
> > > > > closest to your description. It fires as soon as any data is
> > available.
> > > > >
> > > > > Are you sure you don't want some kind of watermark based trigger
> with
> > > > just
> > > > > with a small interval size?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com>
> wrote:
> > > > >
> > > > > > Hi Lukasz,
> > > > > >
> > > > > > Thanks for your response. Is it possible to implement the
> following
> > > > logic
> > > > > > using existing triggers: always fire the trigger on a
> GlobalWindow
> > > > > whenever
> > > > > > watermark advances?
> > > > > >
> > > > > > Shen
> > > > > >
> > > > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik
> > > <lcwik@google.com.invalid
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Users are unable to create custom trigger implementations. If
> you
> > > > tell
> > > > > us
> > > > > > > what you want your trigger to do, we may be able to suggest an
> > > > > > alternative.
> > > > > > >
> > > > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I created a custom trigger class (XYZ) by extending the
> > > > OnceTrigger.
> > > > > > > During
> > > > > > > > execution, I got this error "Cannot translate trigger class
> XYZ
> > > to
> > > > a
> > > > > > > > runner-API proto." It seems that the Triggers.ProtoConverter
> > > class
> > > > > > needs
> > > > > > > to
> > > > > > > > declare a convertSpecific method for my trigger XYZ. How can
> I
> > > use
> > > > my
> > > > > > > > custom trigger without modifying Beam's Triggers class?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Shen
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Kenneth Knowles <kl...@google.com.INVALID>]

Hi Shen,

There are four points (at least) to why we don't currently have custom
triggers:

1. The existing trigger state machine implementations use low-level and
difficult APIs that are not available to users.
2. If we make triggers a UDF then we need to cross the Fn API. It makes a
lot more portability work in terms of engineering and also is really the
center of a tight loop. Instead, runners can very efficiently execute them
directly.
3. There are many state machines that are not valid triggers.
4. Some triggers can be more efficiently executed directly from their
syntax than from a state machine.

Eventually, we may add some kind of triggering UDF. But we may not, because
if you want something like a custom trigger for some use case, then you can
do it with state & timers in ParDo.

But it sounds like you need it for some quite specific testing. Since
"watermark advance event" is a runner-specific concept not part of the
model (a runner doesn't have to actually reify such events) maybe you just
want to test your runner details directly.

Kenn

On Wed, Jun 7, 2017 at 10:22 AM, Shen Li <cs...@gmail.com> wrote:

> Hi Lukasz,
>
> Thanks again for the suggestion. Is there any reason for not allowing users
> create custom triggers?
>
> Shen
>
> On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <lc...@google.com.invalid>
> wrote:
>
> > You should really take a look at TestStream and have runners integrate
> with
> > it instead.
> >
> > There are already several tests which validate TestStream compatible
> > runners to make sure their trigger evaluations are correct.
> >
> > On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com> wrote:
> >
> > > Hi Lukasz,
> > >
> > > Thanks for the suggestion. I am trying to test how the runner generates
> > > watermarks. So I would like to have the trigger to fire on every
> > watermark
> > > advancing event.
> > >
> > > Shen
> > >
> > > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik <lcwik@google.com.invalid
> >
> > > wrote:
> > >
> > > > Look into the AfterPane#elementCountAtLeast trigger as it seems to be
> > the
> > > > closest to your description. It fires as soon as any data is
> available.
> > > >
> > > > Are you sure you don't want some kind of watermark based trigger with
> > > just
> > > > with a small interval size?
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com> wrote:
> > > >
> > > > > Hi Lukasz,
> > > > >
> > > > > Thanks for your response. Is it possible to implement the following
> > > logic
> > > > > using existing triggers: always fire the trigger on a GlobalWindow
> > > > whenever
> > > > > watermark advances?
> > > > >
> > > > > Shen
> > > > >
> > > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik
> > <lcwik@google.com.invalid
> > > >
> > > > > wrote:
> > > > >
> > > > > > Users are unable to create custom trigger implementations. If you
> > > tell
> > > > us
> > > > > > what you want your trigger to do, we may be able to suggest an
> > > > > alternative.
> > > > > >
> > > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I created a custom trigger class (XYZ) by extending the
> > > OnceTrigger.
> > > > > > During
> > > > > > > execution, I got this error "Cannot translate trigger class XYZ
> > to
> > > a
> > > > > > > runner-API proto." It seems that the Triggers.ProtoConverter
> > class
> > > > > needs
> > > > > > to
> > > > > > > declare a convertSpecific method for my trigger XYZ. How can I
> > use
> > > my
> > > > > > > custom trigger without modifying Beam's Triggers class?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Shen
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Shen Li <cs...@gmail.com>]

Hi Ben,

This is only for testing purposes. The runner has its own policy to handle
watermarks. For example, it may decide to drop some watermarks if updates
are too frequent. I was thinking about converting the watermark policy into
an application-visible behavior and use the PAssert to check the result.
So, what comes to mind is to use a global window and a custom trigger that
fires on every watermark advances. So that each watermark becomes an output
tuple. If it is too much trouble, I can work around it by testing in lower
levels.

Thanks,

Shen

On Wed, Jun 7, 2017 at 12:35 PM, Ben Chambers <bc...@google.com.invalid>
wrote:

> There hasn't been a need for user defined triggers and we've found it is
> really hard to create triggers with proper behavior.
>
> Can you elaborate on why you are trying to use triggers to understand
> watermarks in this way? It's not clear how this trigger would be useful to
> that understanding.
>
> On Wed, Jun 7, 2017, 10:22 AM Shen Li <cs...@gmail.com> wrote:
>
> > Hi Lukasz,
> >
> > Thanks again for the suggestion. Is there any reason for not allowing
> users
> > create custom triggers?
> >
> > Shen
> >
> > On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <lc...@google.com.invalid>
> > wrote:
> >
> > > You should really take a look at TestStream and have runners integrate
> > with
> > > it instead.
> > >
> > > There are already several tests which validate TestStream compatible
> > > runners to make sure their trigger evaluations are correct.
> > >
> > > On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com> wrote:
> > >
> > > > Hi Lukasz,
> > > >
> > > > Thanks for the suggestion. I am trying to test how the runner
> generates
> > > > watermarks. So I would like to have the trigger to fire on every
> > > watermark
> > > > advancing event.
> > > >
> > > > Shen
> > > >
> > > > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik
> <lcwik@google.com.invalid
> > >
> > > > wrote:
> > > >
> > > > > Look into the AfterPane#elementCountAtLeast trigger as it seems to
> be
> > > the
> > > > > closest to your description. It fires as soon as any data is
> > available.
> > > > >
> > > > > Are you sure you don't want some kind of watermark based trigger
> with
> > > > just
> > > > > with a small interval size?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com>
> wrote:
> > > > >
> > > > > > Hi Lukasz,
> > > > > >
> > > > > > Thanks for your response. Is it possible to implement the
> following
> > > > logic
> > > > > > using existing triggers: always fire the trigger on a
> GlobalWindow
> > > > > whenever
> > > > > > watermark advances?
> > > > > >
> > > > > > Shen
> > > > > >
> > > > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik
> > > <lcwik@google.com.invalid
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Users are unable to create custom trigger implementations. If
> you
> > > > tell
> > > > > us
> > > > > > > what you want your trigger to do, we may be able to suggest an
> > > > > > alternative.
> > > > > > >
> > > > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I created a custom trigger class (XYZ) by extending the
> > > > OnceTrigger.
> > > > > > > During
> > > > > > > > execution, I got this error "Cannot translate trigger class
> XYZ
> > > to
> > > > a
> > > > > > > > runner-API proto." It seems that the Triggers.ProtoConverter
> > > class
> > > > > > needs
> > > > > > > to
> > > > > > > > declare a convertSpecific method for my trigger XYZ. How can
> I
> > > use
> > > > my
> > > > > > > > custom trigger without modifying Beam's Triggers class?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Shen
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Ben Chambers <bc...@google.com.INVALID>]

There hasn't been a need for user defined triggers and we've found it is
really hard to create triggers with proper behavior.

Can you elaborate on why you are trying to use triggers to understand
watermarks in this way? It's not clear how this trigger would be useful to
that understanding.

On Wed, Jun 7, 2017, 10:22 AM Shen Li <cs...@gmail.com> wrote:

> Hi Lukasz,
>
> Thanks again for the suggestion. Is there any reason for not allowing users
> create custom triggers?
>
> Shen
>
> On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <lc...@google.com.invalid>
> wrote:
>
> > You should really take a look at TestStream and have runners integrate
> with
> > it instead.
> >
> > There are already several tests which validate TestStream compatible
> > runners to make sure their trigger evaluations are correct.
> >
> > On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com> wrote:
> >
> > > Hi Lukasz,
> > >
> > > Thanks for the suggestion. I am trying to test how the runner generates
> > > watermarks. So I would like to have the trigger to fire on every
> > watermark
> > > advancing event.
> > >
> > > Shen
> > >
> > > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik <lcwik@google.com.invalid
> >
> > > wrote:
> > >
> > > > Look into the AfterPane#elementCountAtLeast trigger as it seems to be
> > the
> > > > closest to your description. It fires as soon as any data is
> available.
> > > >
> > > > Are you sure you don't want some kind of watermark based trigger with
> > > just
> > > > with a small interval size?
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com> wrote:
> > > >
> > > > > Hi Lukasz,
> > > > >
> > > > > Thanks for your response. Is it possible to implement the following
> > > logic
> > > > > using existing triggers: always fire the trigger on a GlobalWindow
> > > > whenever
> > > > > watermark advances?
> > > > >
> > > > > Shen
> > > > >
> > > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik
> > <lcwik@google.com.invalid
> > > >
> > > > > wrote:
> > > > >
> > > > > > Users are unable to create custom trigger implementations. If you
> > > tell
> > > > us
> > > > > > what you want your trigger to do, we may be able to suggest an
> > > > > alternative.
> > > > > >
> > > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I created a custom trigger class (XYZ) by extending the
> > > OnceTrigger.
> > > > > > During
> > > > > > > execution, I got this error "Cannot translate trigger class XYZ
> > to
> > > a
> > > > > > > runner-API proto." It seems that the Triggers.ProtoConverter
> > class
> > > > > needs
> > > > > > to
> > > > > > > declare a convertSpecific method for my trigger XYZ. How can I
> > use
> > > my
> > > > > > > custom trigger without modifying Beam's Triggers class?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Shen
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Shen Li <cs...@gmail.com>]

Hi Lukasz,

Thanks again for the suggestion. Is there any reason for not allowing users
create custom triggers?

Shen

On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <lc...@google.com.invalid>
wrote:

> You should really take a look at TestStream and have runners integrate with
> it instead.
>
> There are already several tests which validate TestStream compatible
> runners to make sure their trigger evaluations are correct.
>
> On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com> wrote:
>
> > Hi Lukasz,
> >
> > Thanks for the suggestion. I am trying to test how the runner generates
> > watermarks. So I would like to have the trigger to fire on every
> watermark
> > advancing event.
> >
> > Shen
> >
> > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik <lc...@google.com.invalid>
> > wrote:
> >
> > > Look into the AfterPane#elementCountAtLeast trigger as it seems to be
> the
> > > closest to your description. It fires as soon as any data is available.
> > >
> > > Are you sure you don't want some kind of watermark based trigger with
> > just
> > > with a small interval size?
> > >
> > >
> > >
> > >
> > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com> wrote:
> > >
> > > > Hi Lukasz,
> > > >
> > > > Thanks for your response. Is it possible to implement the following
> > logic
> > > > using existing triggers: always fire the trigger on a GlobalWindow
> > > whenever
> > > > watermark advances?
> > > >
> > > > Shen
> > > >
> > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik
> <lcwik@google.com.invalid
> > >
> > > > wrote:
> > > >
> > > > > Users are unable to create custom trigger implementations. If you
> > tell
> > > us
> > > > > what you want your trigger to do, we may be able to suggest an
> > > > alternative.
> > > > >
> > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I created a custom trigger class (XYZ) by extending the
> > OnceTrigger.
> > > > > During
> > > > > > execution, I got this error "Cannot translate trigger class XYZ
> to
> > a
> > > > > > runner-API proto." It seems that the Triggers.ProtoConverter
> class
> > > > needs
> > > > > to
> > > > > > declare a convertSpecific method for my trigger XYZ. How can I
> use
> > my
> > > > > > custom trigger without modifying Beam's Triggers class?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Shen
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Lukasz Cwik <lc...@google.com.INVALID>]

You should really take a look at TestStream and have runners integrate with
it instead.

There are already several tests which validate TestStream compatible
runners to make sure their trigger evaluations are correct.

On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <cs...@gmail.com> wrote:

> Hi Lukasz,
>
> Thanks for the suggestion. I am trying to test how the runner generates
> watermarks. So I would like to have the trigger to fire on every watermark
> advancing event.
>
> Shen
>
> On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik <lc...@google.com.invalid>
> wrote:
>
> > Look into the AfterPane#elementCountAtLeast trigger as it seems to be the
> > closest to your description. It fires as soon as any data is available.
> >
> > Are you sure you don't want some kind of watermark based trigger with
> just
> > with a small interval size?
> >
> >
> >
> >
> > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com> wrote:
> >
> > > Hi Lukasz,
> > >
> > > Thanks for your response. Is it possible to implement the following
> logic
> > > using existing triggers: always fire the trigger on a GlobalWindow
> > whenever
> > > watermark advances?
> > >
> > > Shen
> > >
> > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik <lcwik@google.com.invalid
> >
> > > wrote:
> > >
> > > > Users are unable to create custom trigger implementations. If you
> tell
> > us
> > > > what you want your trigger to do, we may be able to suggest an
> > > alternative.
> > > >
> > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I created a custom trigger class (XYZ) by extending the
> OnceTrigger.
> > > > During
> > > > > execution, I got this error "Cannot translate trigger class XYZ to
> a
> > > > > runner-API proto." It seems that the Triggers.ProtoConverter class
> > > needs
> > > > to
> > > > > declare a convertSpecific method for my trigger XYZ. How can I use
> my
> > > > > custom trigger without modifying Beam's Triggers class?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Shen
> > > > >
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Shen Li <cs...@gmail.com>]

Hi Lukasz,

Thanks for the suggestion. I am trying to test how the runner generates
watermarks. So I would like to have the trigger to fire on every watermark
advancing event.

Shen

On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik <lc...@google.com.invalid>
wrote:

> Look into the AfterPane#elementCountAtLeast trigger as it seems to be the
> closest to your description. It fires as soon as any data is available.
>
> Are you sure you don't want some kind of watermark based trigger with just
> with a small interval size?
>
>
>
>
> On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com> wrote:
>
> > Hi Lukasz,
> >
> > Thanks for your response. Is it possible to implement the following logic
> > using existing triggers: always fire the trigger on a GlobalWindow
> whenever
> > watermark advances?
> >
> > Shen
> >
> > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik <lc...@google.com.invalid>
> > wrote:
> >
> > > Users are unable to create custom trigger implementations. If you tell
> us
> > > what you want your trigger to do, we may be able to suggest an
> > alternative.
> > >
> > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I created a custom trigger class (XYZ) by extending the OnceTrigger.
> > > During
> > > > execution, I got this error "Cannot translate trigger class XYZ to a
> > > > runner-API proto." It seems that the Triggers.ProtoConverter class
> > needs
> > > to
> > > > declare a convertSpecific method for my trigger XYZ. How can I use my
> > > > custom trigger without modifying Beam's Triggers class?
> > > >
> > > > Thanks,
> > > >
> > > > Shen
> > > >
> > >
> >
>

Re: How to create a custom trigger?

[Lukasz Cwik <lc...@google.com.INVALID>]

Look into the AfterPane#elementCountAtLeast trigger as it seems to be the
closest to your description. It fires as soon as any data is available.

Are you sure you don't want some kind of watermark based trigger with just
with a small interval size?




On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <cs...@gmail.com> wrote:

> Hi Lukasz,
>
> Thanks for your response. Is it possible to implement the following logic
> using existing triggers: always fire the trigger on a GlobalWindow whenever
> watermark advances?
>
> Shen
>
> On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik <lc...@google.com.invalid>
> wrote:
>
> > Users are unable to create custom trigger implementations. If you tell us
> > what you want your trigger to do, we may be able to suggest an
> alternative.
> >
> > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I created a custom trigger class (XYZ) by extending the OnceTrigger.
> > During
> > > execution, I got this error "Cannot translate trigger class XYZ to a
> > > runner-API proto." It seems that the Triggers.ProtoConverter class
> needs
> > to
> > > declare a convertSpecific method for my trigger XYZ. How can I use my
> > > custom trigger without modifying Beam's Triggers class?
> > >
> > > Thanks,
> > >
> > > Shen
> > >
> >
>

Re: How to create a custom trigger?

[Shen Li <cs...@gmail.com>]

Hi Lukasz,

Thanks for your response. Is it possible to implement the following logic
using existing triggers: always fire the trigger on a GlobalWindow whenever
watermark advances?

Shen

On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik <lc...@google.com.invalid>
wrote:

> Users are unable to create custom trigger implementations. If you tell us
> what you want your trigger to do, we may be able to suggest an alternative.
>
> On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com> wrote:
>
> > Hi,
> >
> > I created a custom trigger class (XYZ) by extending the OnceTrigger.
> During
> > execution, I got this error "Cannot translate trigger class XYZ to a
> > runner-API proto." It seems that the Triggers.ProtoConverter class needs
> to
> > declare a convertSpecific method for my trigger XYZ. How can I use my
> > custom trigger without modifying Beam's Triggers class?
> >
> > Thanks,
> >
> > Shen
> >
>

Re: How to create a custom trigger?

[Lukasz Cwik <lc...@google.com.INVALID>]

Users are unable to create custom trigger implementations. If you tell us
what you want your trigger to do, we may be able to suggest an alternative.

On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <cs...@gmail.com> wrote:

> Hi,
>
> I created a custom trigger class (XYZ) by extending the OnceTrigger. During
> execution, I got this error "Cannot translate trigger class XYZ to a
> runner-API proto." It seems that the Triggers.ProtoConverter class needs to
> declare a convertSpecific method for my trigger XYZ. How can I use my
> custom trigger without modifying Beam's Triggers class?
>
> Thanks,
>
> Shen
>