[GitHub] [flink] pnowojski commented on a change in pull request #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins

[GitBox <gi...@apache.org>]

pnowojski commented on a change in pull request #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins
URL: https://github.com/apache/flink/pull/9169#discussion_r309164562
 
 

 ##########
 File path: docs/ops/filesystems/index.md
 ##########
 @@ -83,17 +105,21 @@ fs.hdfs.hadoopconf: /path/to/etc/hadoop
 This registers `/path/to/etc/hadoop` as Hadoop's configuration directory and is where Flink will look for the `core-site.xml` and `hdfs-site.xml` files.
 
 
-## Adding new File System Implementations
+## Adding a new pluggable File System implementation
 
-File systems are represented via the `org.apache.flink.core.fs.FileSystem` class, which captures the ways to access and modify files and objects in that file system. 
-Implementations are discovered by Flink through Java's service abstraction, making it easy to add new file system implementations.
+File systems are represented via the `org.apache.flink.core.fs.FileSystem` class, which captures the ways to access and modify files and objects in that file system.
 
 To add a new file system:
 
   - Add the File System implementation, which is a subclass of `org.apache.flink.core.fs.FileSystem`.
   - Add a factory that instantiates that file system and declares the scheme under which the FileSystem is registered. This must be a subclass of `org.apache.flink.core.fs.FileSystemFactory`.
-  - Add a service entry. Create a file `META-INF/services/org.apache.flink.core.fs.FileSystemFactory` which contains the class name of your file system factory class.
+  - Add a service entry. Create a file `META-INF/services/org.apache.flink.core.fs.FileSystemFactory` which contains the class name of your file system factory class
+  (see the [Java Service Loader docs](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) for more details).
+
+During plugins discovery, the file system factory class will be loaded by a dedicated Java class loader to avoid class conflicts with other plugins and Flink components.
+The same class loader should be used during file system instantiation and the file system operation calls.
 
-See the [Java Service Loader docs](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) for more details on how service loaders work.
+<span class="label label-warning">Warning</span> In practice, it means you should avoid using `Thread.currentThread().getContextClassLoader()` class loader
+in your implementation. 
 
 Review comment:
   Do we want to elaborate on this? That `getContextClassLoader()` can be used inside `org.apache.flink.core.fs.FileSystemFactory#create(URI)`  call stack (however we are discouraging from using it anyway)?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services