[jira] [Comment Edited] (FLINK-11956) Remove shading from filesystems build

["Piotr Nowojski (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873247#comment-16873247 ] 

Piotr Nowojski edited comment on FLINK-11956 at 6/26/19 11:40 AM:
------------------------------------------------------------------

We haven't touched any existing {{FileSystem}} implementation, so yes, they should be loadable without the plugin mechanism. It also should work in the opposite direction.

As long as:
 # there are no un-declared implicit dependencies that were leaking from the user class loader
 # {{FileSystemFactory}} plugin either do not use {{Thread.currentThread().getContextClassLoader()}} or use it only during {{FileSystem}} creation (\{{org.apache.flink.core.fs.FileSystemFactory#create}})

existing file systems should work just as well with plugins. In other words, plugins vs non-plugins {{FileSystem}} they differ only on the packaging & dependencies leaking level, there are no API changes. One caveat here is this {{getContextClassLoader}} issue. Badly written {{FileSystem}} can be inherently incompatible with plugins.

This will be no longer true, once we remove shading. Non shaded {{FileSystems}} should work with plugins by design, but opposite might be no longer true (because of the dependencies clashes). This is why we can not resolve this ticket for 1.9, as this will brake the backwards compatibility.


was (Author: pnowojski):
We haven't touched any existing {{FileSystem}} implementation, so yes, they should be loadable without the plugin mechanism (we have some end to end tests that are using existing non shaded S3 {{FileSystem}}  as plugins). It also should work in the opposite direction. As long as there are not un declared implicit dependencies that were leaking from the user class loader, existing file systems should work just as well with plugins. In other words, plugins vs non-plugins {{FileSystem}} they differ only on the packaging & dependencies leaking level, there are no API changes.

This will be no longer true, once we remove shading. Non shaded \{{FileSystems}} should work with plugins by design, but opposite might be no longer true (because of the dependencies clashes). This is why we can not resolve this ticket for 1.9, as this will brake the backwards compatibility.

> Remove shading from filesystems build
> -------------------------------------
>
>                 Key: FLINK-11956
>                 URL: https://issues.apache.org/jira/browse/FLINK-11956
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Connectors / FileSystem
>    Affects Versions: 1.9.0
>            Reporter: Stefan Richter
>            Assignee: vinoyang
>            Priority: Blocker
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)