You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "cqutwangyu (via GitHub)" <gi...@apache.org> on 2024/01/18 13:44:12 UTC

[I] [Bug] [connector] rocketmq source startOffset error [seatunnel]

cqutwangyu opened a new issue, #6249:
URL: https://github.com/apache/seatunnel/issues/6249

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
   
   
   ### What happened
   
   During program execution, the convertToNextSplit function causes startOffset to be larger than endOffset。
   ![企业微信截图_17055843096927](https://github.com/apache/seatunnel/assets/26395835/51600a6d-2ffb-4c1d-8444-c4929e8e8b85)
   ![企业微信截图_1705584378625](https://github.com/apache/seatunnel/assets/26395835/80743ed5-b4e7-4a4b-9c37-5d0646ce92b7)
   
   ### SeaTunnel Version
   
   2.3.3-release
   
   ### SeaTunnel Config
   
   ```conf
   env {
     execution.parallelism = 1
     job.mode = "STREAMING"
     checkpoint.interval = 30000
   }
   
   source {
     Rocketmq {
       name.srv.addr = "127.0.0.1:9876"
       topics = "SEATUNNEL_TEST"
       result_table_name = "rocketmq_table"
       consumer.group = "SEATUNNEL_TEST_GROUP"
     }
   }
   
   transform {
   }
   
   sink {
     Console {
       source_table_name = "rocketmq_table"
     }
   }
   ```
   
   
   ### Running Command
   
   ```shell
   org.apache.seatunnel.example.engine.SeaTunnelEngineExample
   ```
   
   
   ### Error Exception
   
   ```log
   2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - 
   
   ===============================================================================
   
   
   2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 
   
   2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/seatunnel/issues
   
   2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 
   
   2024-01-18 21:41:56,030 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
   	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
   	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
   	at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
   Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.util.concurrent.CompletionException: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
   For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
   	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
   	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$pollNext$5(RocketMqSourceReader.java:191)
   	at java.lang.Iterable.forEach(Iterable.java:75)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.pollNext(RocketMqSourceReader.java:111)
   	at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
   	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:95)
   	at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
   	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:100)
   	at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
   	at java.util.concurrent.FutureTask.run(FutureTask.java)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   Caused by: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
   For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
   	at org.apache.rocketmq.client.impl.consumer.DefaultLitePullConsumerImpl.seek(DefaultLitePullConsumerImpl.java:592)
   	at org.apache.rocketmq.client.consumer.DefaultLitePullConsumer.seek(DefaultLitePullConsumer.java:286)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$null$4(RocketMqSourceReader.java:126)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqConsumerThread.run(RocketMqConsumerThread.java:58)
   	... 6 more
   
   	at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
   	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184)
   	... 2 more
    
   2024-01-18 21:41:56,030 ERROR org.apache.seatunnel.core.starter.SeaTunnel - 
   ===============================================================================
   
   
   
   Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
   	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
   	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
   	at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
   Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.util.concurrent.CompletionException: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
   For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
   	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
   	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$pollNext$5(RocketMqSourceReader.java:191)
   	at java.lang.Iterable.forEach(Iterable.java:75)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.pollNext(RocketMqSourceReader.java:111)
   	at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
   	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:95)
   	at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
   	at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:100)
   	at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
   	at java.util.concurrent.FutureTask.run(FutureTask.java)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   Caused by: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
   For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
   	at org.apache.rocketmq.client.impl.consumer.DefaultLitePullConsumerImpl.seek(DefaultLitePullConsumerImpl.java:592)
   	at org.apache.rocketmq.client.consumer.DefaultLitePullConsumer.seek(DefaultLitePullConsumer.java:286)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$null$4(RocketMqSourceReader.java:126)
   	at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqConsumerThread.run(RocketMqConsumerThread.java:58)
   	... 6 more
   
   	at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
   	at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184)
   	... 2 more
   ```
   
   
   ### Zeta or Flink or Spark Version
   
   2.3.3-release
   
   ### Java or Scala Version
   
   1.8.0_311
   
   ### Screenshots
   
   ![image](https://github.com/apache/seatunnel/assets/26395835/c8c95f9c-9375-4a23-bf2e-44dbf1015920)
   
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [connector] rocketmq source startOffset error [seatunnel]

Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 closed issue #6249: [Bug] [connector] rocketmq source startOffset error
URL: https://github.com/apache/seatunnel/issues/6249


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org