You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "cqutwangyu (via GitHub)" <gi...@apache.org> on 2024/01/18 13:44:12 UTC
[I] [Bug] [connector] rocketmq source startOffset error [seatunnel]
cqutwangyu opened a new issue, #6249:
URL: https://github.com/apache/seatunnel/issues/6249
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
During program execution, the convertToNextSplit function causes startOffset to be larger than endOffset。


### SeaTunnel Version
2.3.3-release
### SeaTunnel Config
```conf
env {
execution.parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 30000
}
source {
Rocketmq {
name.srv.addr = "127.0.0.1:9876"
topics = "SEATUNNEL_TEST"
result_table_name = "rocketmq_table"
consumer.group = "SEATUNNEL_TEST_GROUP"
}
}
transform {
}
sink {
Console {
source_table_name = "rocketmq_table"
}
}
```
### Running Command
```shell
org.apache.seatunnel.example.engine.SeaTunnelEngineExample
```
### Error Exception
```log
2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error,
2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/seatunnel/issues
2024-01-18 21:41:56,028 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed
2024-01-18 21:41:56,030 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.util.concurrent.CompletionException: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$pollNext$5(RocketMqSourceReader.java:191)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.pollNext(RocketMqSourceReader.java:111)
at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:95)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:100)
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at org.apache.rocketmq.client.impl.consumer.DefaultLitePullConsumerImpl.seek(DefaultLitePullConsumerImpl.java:592)
at org.apache.rocketmq.client.consumer.DefaultLitePullConsumer.seek(DefaultLitePullConsumer.java:286)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$null$4(RocketMqSourceReader.java:126)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqConsumerThread.run(RocketMqConsumerThread.java:58)
... 6 more
at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184)
... 2 more
2024-01-18 21:41:56,030 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.util.concurrent.CompletionException: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$pollNext$5(RocketMqSourceReader.java:191)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.pollNext(RocketMqSourceReader.java:111)
at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:95)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:100)
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:613)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.rocketmq.client.exception.MQClientException: Seek offset illegal, seek offset = 4, min offset = 0, max offset = 3
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at org.apache.rocketmq.client.impl.consumer.DefaultLitePullConsumerImpl.seek(DefaultLitePullConsumerImpl.java:592)
at org.apache.rocketmq.client.consumer.DefaultLitePullConsumer.seek(DefaultLitePullConsumer.java:286)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqSourceReader.lambda$null$4(RocketMqSourceReader.java:126)
at org.apache.seatunnel.connectors.seatunnel.rocketmq.source.RocketMqConsumerThread.run(RocketMqConsumerThread.java:58)
... 6 more
at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:184)
... 2 more
```
### Zeta or Flink or Spark Version
2.3.3-release
### Java or Scala Version
1.8.0_311
### Screenshots

### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [connector] rocketmq source startOffset error [seatunnel]
Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 closed issue #6249: [Bug] [connector] rocketmq source startOffset error
URL: https://github.com/apache/seatunnel/issues/6249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org