You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "antlers-lv (via GitHub)" <gi...@apache.org> on 2024/07/15 09:16:32 UTC

[I] [Bug] [seatunnel-plugin-discovery] Use error reporting in fatjar mode [seatunnel]

antlers-lv opened a new issue, #7207:
URL: https://github.com/apache/seatunnel/issues/7207

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
   
   
   ### What happened
   
   After I changed the package of seatunnel-web from shade to fatjar, I started seatunnel-web as jar package, and the following problems occurred
   ``` java
   Caused by: java.lang.NoClassDefFoundError: org/apache/seatunnel/api/table/factory/TableSinkFactory
           at java.lang.ClassLoader.defineClass1(Native Method)
           at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
           at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
           at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
           at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
           at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
           at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
           at java.security.AccessController.doPrivileged(Native Method)
           at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
           at java.lang.Class.forName0(Native Method)
           at java.lang.Class.forName(Class.java:348)
           at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
           at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
           at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
           at java.util.Iterator.forEachRemaining(Iterator.java:116)
           at org.apache.seatunnel.api.table.factory.FactoryUtil.discoverFactories(FactoryUtil.java:242)
           at org.apache.seatunnel.app.thirdparty.framework.PluginDiscoveryUtil.getConnectorFeatures(PluginDiscoveryUtil.java:70)
           at org.apache.seatunnel.app.bean.connector.ConnectorCache.syncSourceFeature(ConnectorCache.java:136)
           at org.apache.seatunnel.app.bean.connector.ConnectorCache.refresh(ConnectorCache.java:132)
           at org.apache.seatunnel.app.bean.connector.ConnectorCache.<init>(ConnectorCache.java:73)
           at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
           at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
           at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
           at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
           at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
           ... 27 common frames omitted
   Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.api.table.factory.TableSinkFactory
           at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
           ... 54 common frames omitted
   ```
   The problem I found is that seatunnel-plugin-discovery uses the URLClassLoader to find all connector extensions, but does not specify the parent for the URLClassLoader. Meanwhile, the current context uses Fatjar's LaunchedURLClassLOader, and the parent in URLClassLoader is set to the default application classloader Launcher@AppClassLoader. As a result, the Seatunnel-APi-related Class introduced into the jar package was not found when the SPI Load was performed.
   
   Finally, I think the URLClassloader should be built with the ClassLoader used by the main line as the parent. Of course I tried and solved the problem.
   
   such as  getAllPlugin method of AbstractPluginDiscovery
   
   ```java
   public Map<PluginType, LinkedHashMap<PluginIdentifier, OptionRule>> getAllPlugin()
               throws IOException {
           List<Factory> factories;
           if (this.pluginDir.toFile().exists()) {
               log.info("load plugin from plugin dir: {}", this.pluginDir);
               List<URL> files = FileUtils.searchJarFiles(this.pluginDir);
               factories =
                       FactoryUtil.discoverFactories(new URLClassLoader(files.toArray(new URL[0]), Thread.currentThread().getContextClassLoader()));
           } else {
               log.warn("plugin dir: {} not exists, load plugin from classpath", this.pluginDir);
               factories =
                       FactoryUtil.discoverFactories(Thread.currentThread().getContextClassLoader());
           }
          ... 
   
   ```
   
   ### SeaTunnel Version
   
   2.3.3
   
   ### SeaTunnel Config
   
   ```conf
   Not applicable
   ```
   
   
   ### Running Command
   
   ```shell
   Not applicable
   ```
   
   
   ### Error Exception
   
   ```log
   Not applicable
   ```
   
   
   ### Zeta or Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   _No response_
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [seatunnel-plugin-discovery] Use error reporting in fatjar mode [seatunnel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7207:
URL: https://github.com/apache/seatunnel/issues/7207#issuecomment-2303341033

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [seatunnel-plugin-discovery] Use error reporting in fatjar mode [seatunnel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7207:
URL: https://github.com/apache/seatunnel/issues/7207#issuecomment-2290147253

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [seatunnel-plugin-discovery] Use error reporting in fatjar mode [seatunnel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #7207: [Bug] [seatunnel-plugin-discovery] Use error reporting in fatjar mode
URL: https://github.com/apache/seatunnel/issues/7207


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org