You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/03/23 01:11:27 UTC
[GitHub] [incubator-druid] jihoonson opened a new issue #7330: Rolling
update failure in Kafka indexing service
jihoonson opened a new issue #7330: Rolling update failure in Kafka indexing service
URL: https://github.com/apache/incubator-druid/issues/7330
### Affected Version
0.14.0-rc2
### Description
[When publishing a segment, the start offsets of the new sequence is matched to the end offsets of the previous sequence](https://github.com/apache/incubator-druid/blob/master/server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java#L889). Here, the start offsets must contain `SeekableStreamStartSequenceNumbers`. However, the Kafka tasks of an old version would send `SeekableStreamEndSequenceNumbers` because it's the default implementation of `SeekableStreamSequenceNumbers`.
Here is the stack trace.
```
Caused by: org.skife.jdbi.v2.exceptions.CallbackFailedException: org.apache.druid.java.util.common.IAE: Expected instance of org.apache.druid.indexing.seekables
tream.SeekableStreamEndSequenceNumbers, got org.apache.druid.indexing.seekablestream.SeekableStreamStartSequenceNumbers
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:284) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.inTransaction(DBI.java:353) ~[jdbi-2.63.1.jar:2.63.1]
at org.apache.druid.metadata.SQLMetadataConnector.lambda$retryTransaction$1(SQLMetadataConnector.java:148) ~[druid-server-0.14.0-iap-pre5.jar:0.14.0-iap
-pre5]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86) ~[druid-core-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114) ~[druid-core-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.SQLMetadataConnector.retryTransaction(SQLMetadataConnector.java:148) ~[druid-server-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator.announceHistoricalSegments(IndexerSQLMetadataStorageCoordinator.java:285) ~[druid-serv
er-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.lambda$perform$0(SegmentTransactionalInsertAction.java:115) ~[druid-indexin
g-service-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.overlord.CriticalAction.perform(CriticalAction.java:53) ~[druid-indexing-service-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.overlord.TaskLockbox.doInCriticalSection(TaskLockbox.java:627) ~[druid-indexing-service-0.14.0-iap-pre5.jar:0.14.0-iap-pre5
]
at org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.perform(SegmentTransactionalInsertAction.java:110) ~[druid-indexing-service
-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
... 75 more
Caused by: org.apache.druid.java.util.common.IAE: Expected instance of org.apache.druid.indexing.seekablestream.SeekableStreamEndSequenceNumbers, got org.apache
.druid.indexing.seekablestream.SeekableStreamStartSequenceNumbers
at org.apache.druid.indexing.seekablestream.SeekableStreamEndSequenceNumbers.plus(SeekableStreamEndSequenceNumbers.java:132) ~[druid-indexing-service-0.
14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.seekablestream.SeekableStreamDataSourceMetadata.plus(SeekableStreamDataSourceMetadata.java:77) ~[druid-indexing-service-0.1
4.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.seekablestream.SeekableStreamDataSourceMetadata.matches(SeekableStreamDataSourceMetadata.java:59) ~[druid-indexing-service-
0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator.updateDataSourceMetadataWithHandle(IndexerSQLMetadataStorageCoordinator.java:890) ~[dr
uid-server-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator$1.inTransaction(IndexerSQLMetadataStorageCoordinator.java:300) ~[druid-server-0.14.0-i
ap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator$1.inTransaction(IndexerSQLMetadataStorageCoordinator.java:287) ~[druid-server-0.14.0-i
ap-pre5.jar:0.14.0-iap-pre5]
at org.skife.jdbi.v2.tweak.transactions.LocalTransactionHandler.inTransaction(LocalTransactionHandler.java:184) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.tweak.transactions.LocalTransactionHandler.inTransaction(LocalTransactionHandler.java:216) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.BasicHandle.inTransaction(BasicHandle.java:350) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI$7.withHandle(DBI.java:357) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:281) ~[jdbi-2.63.1.jar:2.63.1]
at org.skife.jdbi.v2.DBI.inTransaction(DBI.java:353) ~[jdbi-2.63.1.jar:2.63.1]
at org.apache.druid.metadata.SQLMetadataConnector.lambda$retryTransaction$1(SQLMetadataConnector.java:148) ~[druid-server-0.14.0-iap-pre5.jar:0.14.0-iap
-pre5]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:86) ~[druid-core-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:114) ~[druid-core-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.SQLMetadataConnector.retryTransaction(SQLMetadataConnector.java:148) ~[druid-server-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator.announceHistoricalSegments(IndexerSQLMetadataStorageCoordinator.java:285) ~[druid-serv
er-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.lambda$perform$0(SegmentTransactionalInsertAction.java:115) ~[druid-indexin
g-service-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.overlord.CriticalAction.perform(CriticalAction.java:53) ~[druid-indexing-service-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
at org.apache.druid.indexing.overlord.TaskLockbox.doInCriticalSection(TaskLockbox.java:627) ~[druid-indexing-service-0.14.0-iap-pre5.jar:0.14.0-iap-pre5
]
at org.apache.druid.indexing.common.actions.SegmentTransactionalInsertAction.perform(SegmentTransactionalInsertAction.java:110) ~[druid-indexing-service
-0.14.0-iap-pre5.jar:0.14.0-iap-pre5]
... 75 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org