You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "shunping (via GitHub)" <gi...@apache.org> on 2025/09/12 00:33:56 UTC
[PR] [Prism] Support after-process-time triggers - part 1 [beam]
shunping opened a new pull request, #36126:
URL: https://github.com/apache/beam/pull/36126
Construct after-processing-time trigger from proto and define trigger callbacks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support after-process-time triggers - part 1 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36126:
URL: https://github.com/apache/beam/pull/36126#issuecomment-3283313798
Assigning reviewers:
R: @lostluck for label go.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support AfterProcessingTime triggers - part 1 [beam]
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #36126:
URL: https://github.com/apache/beam/pull/36126#issuecomment-3308973907
Sorry, this fell off my radar. Looking again today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support after-process-time triggers - part 1 [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #36126:
URL: https://github.com/apache/beam/pull/36126#discussion_r2342796607
##########
sdks/go/pkg/beam/runners/prism/internal/engine/strategy.go:
##########
@@ -573,4 +574,105 @@ func (t *TriggerDefault) String() string {
return "Default"
}
-// TODO https://github.com/apache/beam/issues/31438 Handle TriggerAfterProcessingTime
+// TimestampTransform is the engine's representation of a processing time transform.
+type TimestampTransform struct {
+ Delay time.Duration
+ AlignToPeriod time.Duration
+ AlignToOffset time.Duration
+}
+
+// TriggerAfterProcessingTime fires once after a specified amount of processing time
+// has passed since an element was first seen.
+// Uses the extra state field to track the processing time of the first element.
+type TriggerAfterProcessingTime struct {
+ Transforms []TimestampTransform
+}
+
+type afterProcessingTimeState struct {
+ emNow mtime.Time
+ firingTime mtime.Time
+ endOfWindowReached bool
Review Comment:
Alternatively, we can pass `triggerInput` as an argument in shouldFire() and reset(). Then we don't need to put emNow and endOfWindowReached inside `ts.extra`.
I am fine with both approaches, but would also want to see if you have any opinion on that. @lostluck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support AfterProcessingTime triggers - part 1 [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #36126:
URL: https://github.com/apache/beam/pull/36126#issuecomment-3330700716
gentle ping here @lostluck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support AfterProcessingTime triggers - part 1 [beam]
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on code in PR #36126:
URL: https://github.com/apache/beam/pull/36126#discussion_r2360630907
##########
sdks/go/pkg/beam/runners/prism/internal/engine/strategy.go:
##########
@@ -573,4 +574,105 @@ func (t *TriggerDefault) String() string {
return "Default"
}
-// TODO https://github.com/apache/beam/issues/31438 Handle TriggerAfterProcessingTime
+// TimestampTransform is the engine's representation of a processing time transform.
+type TimestampTransform struct {
+ Delay time.Duration
+ AlignToPeriod time.Duration
+ AlignToOffset time.Duration
+}
+
+// TriggerAfterProcessingTime fires once after a specified amount of processing time
+// has passed since an element was first seen.
+// Uses the extra state field to track the processing time of the first element.
+type TriggerAfterProcessingTime struct {
+ Transforms []TimestampTransform
+}
+
+type afterProcessingTimeState struct {
+ emNow mtime.Time
+ firingTime mtime.Time
+ endOfWindowReached bool
Review Comment:
It's handy to have emNow in state, since it helps track the change happening. Feels consistent. That's partly why there isn't the triggerInput on those methods presently: it's harder to reason about the state machine, if the state isn't in the machine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support AfterProcessingTime triggers - part 1 [beam]
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on PR #36126:
URL: https://github.com/apache/beam/pull/36126#issuecomment-3330784464
I don't think I got an email for the emoji response to my prior comment, or the move away from draft... Thank you for your patience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support AfterProcessingTime triggers - part 1 [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #36126:
URL: https://github.com/apache/beam/pull/36126#issuecomment-3330786697
Because the actual trigger is not supported yet. Need one or two more PRs :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support after-process-time triggers - part 1 [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #36126:
URL: https://github.com/apache/beam/pull/36126#discussion_r2342796607
##########
sdks/go/pkg/beam/runners/prism/internal/engine/strategy.go:
##########
@@ -573,4 +574,105 @@ func (t *TriggerDefault) String() string {
return "Default"
}
-// TODO https://github.com/apache/beam/issues/31438 Handle TriggerAfterProcessingTime
+// TimestampTransform is the engine's representation of a processing time transform.
+type TimestampTransform struct {
+ Delay time.Duration
+ AlignToPeriod time.Duration
+ AlignToOffset time.Duration
+}
+
+// TriggerAfterProcessingTime fires once after a specified amount of processing time
+// has passed since an element was first seen.
+// Uses the extra state field to track the processing time of the first element.
+type TriggerAfterProcessingTime struct {
+ Transforms []TimestampTransform
+}
+
+type afterProcessingTimeState struct {
+ emNow mtime.Time
+ firingTime mtime.Time
+ endOfWindowReached bool
Review Comment:
Alternatively, we can pass `triggerInput` as an argument in `shouldFire()` and `reset()`. Then we don't need to put `emNow` and `endOfWindowReached` inside `ts.extra`.
I am fine with both approaches, but would also want to see if you have any opinion on that. @lostluck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Prism] Support AfterProcessingTime triggers - part 1 [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping merged PR #36126:
URL: https://github.com/apache/beam/pull/36126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org