You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Reza ardeshir rokni (Jira)" <ji...@apache.org> on 2021/03/15 03:30:00 UTC
[jira] [Commented] (BEAM-2953) Timeseries processing extensions
using state API
[ https://issues.apache.org/jira/browse/BEAM-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17301358#comment-17301358 ]
Reza ardeshir rokni commented on BEAM-2953:
-------------------------------------------
This is being worked on in sample library external to beam repo:
https://github.com/GoogleCloudPlatform/dataflow-sample-applications/tree/master/timeseries-streaming
> Timeseries processing extensions using state API
> ------------------------------------------------
>
> Key: BEAM-2953
> URL: https://issues.apache.org/jira/browse/BEAM-2953
> Project: Beam
> Issue Type: Improvement
> Components: sdk-ideas
> Affects Versions: 2.7.0
> Reporter: Reza ardeshir rokni
> Priority: P3
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> A general set of timeseries transforms that abstract the user from the process of dealing with some of the common problems when dealing with timeseries using BEAM (in stream or batch mode).
> BEAM can be used to build out some very interesting pre-processing stages for time series data. Some examples that will be useful:
> - Downsampling time series based on simple MIN, MAX, COUNT, SUM, LAST, FIRST
> - Creating a value for each downsampled window even if no value has been emitted for the specific key.
> - Loading the value of a downsample with the previous value (used in FX with previous close being brought into current open value)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)