Statefun remote functions - acessing kafka headers from a remote function

[Filip Karnicki <fi...@gmail.com>]

Hi, is there a neat way to access kafka headers from within a remote
function without using the datastream api to insert the headers as part of
a RoutableMessage payload?

Many thanks
Fil

Re: Statefun remote functions - acessing kafka headers from a remote function

[Igal Shilman <ig...@apache.org>]

Hi Fil,

The default Kafka ingress that ships with StateFun indeed doesn't bundle
the headers with the incoming message, so there is no way of getting them
at the moment, without doing some work :(
I'd be also happy to kick off the discussion (I guess JIRA would be the
right place) about supporting this feature.

The way I'd approach that at the moment is by adding a custom ingress via
the embedded sdk, unfortunately this might be a little involved
but I had this example project [1] for an older version of StateFun that
you can use as a starting point, if you are interested I can point you to
the changes required for it to work with the 3.x branch in subsequent
emails.

The most relevant part for you would be here [3][4]

The resulting artifact can be included within your StateFun application and
will make this new ingress available at the runtime [2]

[1] https://github.com/igalshilman/custom-ingress/
[2]
https://nightlies.apache.org/flink/flink-statefun-docs-master/docs/modules/embedded/
[3]
https://github.com/igalshilman/custom-ingress/blob/main/src/main/java/com/igal/Module.java#L76
[4]
https://github.com/igalshilman/custom-ingress/blob/main/src/main/protobuf/messages.proto#L26

I hope this helps,
Igal.


On Tue, Nov 2, 2021 at 2:04 PM Filip Karnicki <fi...@gmail.com>
wrote:

> Hi, is there a neat way to access kafka headers from within a remote
> function without using the datastream api to insert the headers as part of
> a RoutableMessage payload?
>
> Many thanks
> Fil
>