You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@inlong.apache.org by "wangpeix (via GitHub)" <gi...@apache.org> on 2023/03/31 14:35:31 UTC

[GitHub] [inlong] wangpeix opened a new issue, #7752: [Bug][Agent] PulsarSink threadPool throw reject exception

wangpeix opened a new issue, #7752:
URL: https://github.com/apache/inlong/issues/7752

   ### What happened
   
   When remove the file data source on databoard. Then add one file data source in new inlong group on databoard. Agent occur the following exception:
   ```java
   023-03-31 22:06:50.408 -ERROR  [               task-job_11_2-6] o.a.i.a.c.t.TaskWrapper       :211 error while running wrapper
   java.util.concurrent.RejectedExecutionException: Task org.apache.inlong.agent.plugin.sinks.PulsarSink$$Lambda$208/701747533@5f91f266 rejected from java.util.concurrent.ThreadPoolExecutor@43d6a181[Shutting down, pool size = 6, active threads = 6, queued tasks = 0, completed tasks = 2]
   	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) ~[?:1.8.0_151]
   	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) ~[?:1.8.0_151]
   	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) ~[?:1.8.0_151]
   	at org.apache.inlong.agent.plugin.sinks.PulsarSink.init(PulsarSink.java:153) ~[agent-plugins-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
   	at org.apache.inlong.agent.core.task.Task.init(Task.java:76) ~[agent-core-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
   	at org.apache.inlong.agent.core.task.TaskWrapper.run(TaskWrapper.java:203) ~[agent-core-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_151]
   	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_151]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_151]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_151]
   	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_151]
   ``` 
   
   ### What you expected to happen
   
   Agent `PulsarSink` thread pool is global, like this:
   ```java
   public class PulsarSink extends AbstractSink {
   
       private static final Logger LOGGER = LoggerFactory.getLogger(LoggerFactory.class);
       private static final AtomicInteger CLIENT_INDEX = new AtomicInteger(0);
       private static final ExecutorService EXECUTOR_SERVICE = new ThreadPoolExecutor(0, Integer.MAX_VALUE,
               60L, TimeUnit.SECONDS, new SynchronousQueue<>(), new AgentThreadFactory("PulsarSink"));
   ```
   
   When `PulsarSink#destroy`, the thread pool will be shut down.
   ```java
   public void destroy() {
     ...
     EXECUTOR_SERVICE.shutdown();
     ...
   }
   ```
   
   Then `PulsarSink#init`  submit thread to the `shutdown thread pool`  may cause the problem.   
   ```java
   public void init(JobProfile jobConf) {
           ...
           EXECUTOR_SERVICE.execute(sendDataThread());
           ...
   }
   ```
   
   
   ### How to reproduce
   
   Remove the file data source on databoard. Then add one file data source in new inlong group on databoard. Agent occur the exception.
   
   ### Environment
   
   _No response_
   
   ### InLong version
   
   master
   
   ### InLong Component
   
   InLong Agent
   
   ### Are you willing to submit PR?
   
   - [ ] Yes, I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [inlong] dockerzhang closed issue #7752: [Bug][Agent] PulsarSink threadPool throw reject exception

Posted by "dockerzhang (via GitHub)" <gi...@apache.org>.
dockerzhang closed issue #7752: [Bug][Agent] PulsarSink threadPool throw reject exception
URL: https://github.com/apache/inlong/issues/7752


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org