You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by fhueske <gi...@git.apache.org> on 2015/11/04 21:54:34 UTC

[GitHub] flink pull request: [FLINK-2940] [JavaAPI] Remove Chill dependency...

GitHub user fhueske opened a pull request:

    https://github.com/apache/flink/pull/1330

    [FLINK-2940] [JavaAPI] Remove Chill dependency from flink-java.

    This PR replaces `flink-java`'s `chill` dependency by `chill-java` to decouple `flink-java` from Scala.
    Chill is used within the `KryoSerializer` to obtain a "Scala-preconfigured" Kryo instance. Hence, Chill is still required for Scala DataSet or DataStream programs. Separate KryoSerializers for Scala and Java are hard to provide, because `GenericTypeInformation` can be generated through the `TypeExtractor`, `TypeParser`, and the Scala compiler marko. 
    
    In this PR, the `KryoSerializer` tries to dynamically load Chill's `ScalaKryoInstantiator` and generate a Kryo instance through reflection to circumvent the Chill dependency. As a fallback, a regular Kryo instance is created. Consequently, Chill's Kryo instance is used whenever Chill is present in the classpath. To ensure that client, JM, and TM have Chill in their classpath, we add Chill as a dependency to `flink-client`.
    
    **Note**: Before merging, we need to check if this works for Java and Scala programs which are submitted to a cluster (including some collection data sources which are serialized at the client).

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/fhueske/flink scalaFlinkJava

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1330.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1330
    
----
commit 9b3b86fb8bbd1d36914b1f13519e314090ce9deb
Author: Fabian Hueske <fh...@apache.org>
Date:   2015-11-03T15:57:49Z

    [FLINK-2940] [JavaAPI] Remove Chill dependency from flink-java.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2972] [JavaAPI] Remove Chill dependency...

Posted by fhueske <gi...@git.apache.org>.

Github user fhueske commented on the pull request:

    https://github.com/apache/flink/pull/1330#issuecomment-153865423
  
    Sorry, I linked the PR to the wrong JIRA issue. 
    Will close this PR and open a new one.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2972] [JavaAPI] Remove Chill dependency...

Posted by fhueske <gi...@git.apache.org>.

Github user fhueske closed the pull request at:

    https://github.com/apache/flink/pull/1330


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---