[jira] [Comment Edited] (KAFKA-8629) Kafka Streams Apps to support small native images through GraalVM

["Andy Muir (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878376#comment-16878376 ] 

Andy Muir edited comment on KAFKA-8629 at 7/4/19 4:42 PM:
----------------------------------------------------------

Kafka fork with modified code: [https://github.com/muirandy/kafka/tree/graal-native-image-support]

Shows whats needed to make Kafka Streams app work with GraalVM native image.

Note: More changes may be required, and as stated in the description above, this code is not ready for a PR.

NB: This is my original work and I license the work to the project under the project's open source license.


was (Author: muirandy):
Kafka fork with modified code: [https://github.com/muirandy/kafka/tree/graal-native-image-support]

Shows whats needed to make Kafka Streams app work with GraalVM native image.

Note: More changes may be required, and as stated in the description above, this code is not ready for a PR.

> Kafka Streams Apps to support small native images through GraalVM
> -----------------------------------------------------------------
>
>                 Key: KAFKA-8629
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8629
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions: 2.3.0
>         Environment: OSX
> Linux on Docker
>            Reporter: Andy Muir
>            Priority: Minor
>
> I'm investigating using [GraalVM|http://example.com/] to help with reducing docker image size and required resources for a simple Kafka Streams microservice. To this end, I'm looking at running a microservice which:
> 1) consumes from a Kafka topic (XML)
> 2) Transforms into JSON
> 3) Produces to a new Kafka topic.
> The Kafka Streams app running in the JVM works fine.
> When I attempt to build it to a GraalVM native image (binary executable which does not require the JVM, hence smaller image size and less resources), I encountered a few [incompatibilities|https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md] with the source code in Kafka.
> I've implemented a workaround for each of these in a fork (link to follow) to help establish if it is feasible. I don't intend (at this stage) for the changes to be applied to the broker - I'm only after Kafka Streams for now. I'm not sure whether it'd be a good idea for the broker itself to run as a native image!
> There were 2 issues getting the native image with kafka streams:
> 1) Some Reflection use cases using MethodHandle
> 2) Anything JMX
> To work around these issues, I have:
> 1) Replaced use of MethodHandle with alternatives
> 2) Commented out the JMX code in a few places
> While the first may be sustainable, I'd expect that the 2nd option should be put behind a configuration switch to allow the existing code to be used by default and turning off JMX if configured.
> *I haven't created a PR for now, as I'd like feedback to decide if it is going to be feasible to carry this forwards.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)