You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Satish Saley <sa...@apache.org> on 2021/06/03 18:28:54 UTC
Flattening of events
Hi team,
I am trying to figure out a way to flatten events in my Flink app.
Event that i am consuming from Kafka is
UpperLevelData {
int upperId;
List<ModuleData> listOfModules
}
ModuleData {
int moduleId;
string info;
}
After consuming this event, i want to flatten it out in following format -
FlinkReshapedData{
int upperId;
int moduleId;
string info;
}
How would I do it? Shall I use process function for it
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/
?
Re: Flattening of events
Posted by Chesnay Schepler <ch...@apache.org>.
Have a look at flatMaps:
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/overview/#datastream-rarr-datastream-1
On 6/3/2021 8:28 PM, Satish Saley wrote:
> Hi team,
> I am trying to figure out a way to flatten events in my Flink app.
>
> Event that i am consuming from Kafka is
>
> UpperLevelData {
> int upperId;
> List<ModuleData> listOfModules
> }
>
> ModuleData {
> int moduleId;
> string info;
> }
>
> After consuming this event, i want to flatten it out in following format -
>
> FlinkReshapedData{
> int upperId;
> int moduleId;
> string info;
> }
>
> How would I do it? Shall I use process function for it
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/
> <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/>
> ?
Re: Flattening of events
Posted by Chesnay Schepler <ch...@apache.org>.
Have a look at flatMaps:
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/overview/#datastream-rarr-datastream-1
On 6/3/2021 8:28 PM, Satish Saley wrote:
> Hi team,
> I am trying to figure out a way to flatten events in my Flink app.
>
> Event that i am consuming from Kafka is
>
> UpperLevelData {
> int upperId;
> List<ModuleData> listOfModules
> }
>
> ModuleData {
> int moduleId;
> string info;
> }
>
> After consuming this event, i want to flatten it out in following format -
>
> FlinkReshapedData{
> int upperId;
> int moduleId;
> string info;
> }
>
> How would I do it? Shall I use process function for it
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/
> <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/>
> ?