You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/05/12 00:32:04 UTC
[jira] [Commented] (KAFKA-3487) Support per-connector/per-task
classloaders in Connect
[ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007425#comment-16007425 ]
ASF GitHub Bot commented on KAFKA-3487:
---------------------------------------
GitHub user kkonstantine opened a pull request:
https://github.com/apache/kafka/pull/3028
KAFKA-3487: Support classloading isolation in Connect.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kkonstantine/kafka KAFKA-3487-Support-classloading-isolation-in-Connect
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3028.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3028
----
commit 6e3e92cfadfa7c03090476a0f025b5a6b7989d5c
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-05T00:00:29Z
KAFKA-3487: Support classloading isolation in Connect.
* Add module.path in worker config.
commit 84967f43712dd55523ca33e855fa0252f9cdb677
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-09T23:10:08Z
Add isolation package.
* Add a delegating class loader
* Add per module class loaders
* Add module factories
commit 7e1169b081c451d0f4a3920e75e0bf1961ee84e8
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-12T00:24:08Z
Add config property only for standalone currently.
commit d8cd6330c6304267a5d7b07c8ef48aa201d95ac8
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-09T23:19:24Z
Add maven-artifact dependency for module versioning.
commit 50c8a1b79366ce04231b7c88b9abc36871afc143
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-09T23:21:15Z
Replace connector factory with new module factory.
commit df22a391e920b2f24ad795df929968cd0d5e2dd1
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-11T18:19:39Z
Consolidating Modules factory class.
commit ba3a78dbcf10a94a2080d7f7ef87585826c27ea8
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-05-12T00:20:41Z
Setting thread context class loader to a modules loader.
----
> Support per-connector/per-task classloaders in Connect
> ------------------------------------------------------
>
> Key: KAFKA-3487
> URL: https://issues.apache.org/jira/browse/KAFKA-3487
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect
> Affects Versions: 0.10.0.0
> Reporter: Ewen Cheslack-Postava
> Assignee: Liquan Pei
> Priority: Critical
> Labels: needs-kip
>
> Currently we just use the default ClassLoader in Connect. However, this limits how we can compatibly load conflicting connector plugins. Ideally we would use a separate class loader per connector/task that is instantiated to avoid potential conflicts.
> Note that this also opens up options for other ways to provide jars to instantiate connectors. For example, Spark uses this to dynamically publish classes defined in the REPL and load them via URL: https://ardoris.wordpress.com/2014/03/30/how-spark-does-class-loading/ But much simpler examples (include URL in the connector class instead of just class name) are also possible and could be a nice way to more support dynamic sets of connectors, multiple versions of the same connector, etc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)