You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/01/05 12:17:54 UTC
[GitHub] [incubator-pinot] KKcorps opened a new pull request #6410: Bug fixes: Handle connection broken exception in Kinesis
KKcorps opened a new pull request #6410:
URL: https://github.com/apache/incubator-pinot/pull/6410
Currently, if a connection is broken kinesis returns 0 records instead of returning the records read before the connection.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on a change in pull request #6410: Add unit tests in Kinesis consumer
Posted by GitBox <gi...@apache.org>.
npawar commented on a change in pull request #6410:
URL: https://github.com/apache/incubator-pinot/pull/6410#discussion_r566992199
##########
File path: pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProvider.java
##########
@@ -83,7 +91,7 @@ public long fetchPartitionOffset(@Nonnull OffsetCriteria offsetCriteria, long ti
List<PartitionGroupInfo> newPartitionGroupInfos = new ArrayList<>();
Map<String, Shard> shardIdToShardMap =
- _kinesisConnectionHandler.getShards().stream().collect(Collectors.toMap(Shard::shardId, s -> s));
+ _kinesisConnectionHandler.getShards().stream().collect(Collectors.toMap(Shard::shardId, s -> s, (s1, s2) -> s1));
Review comment:
why would a collision happen here?
##########
File path: pinot-plugins/pinot-stream-ingestion/pinot-kinesis/pom.xml
##########
@@ -36,9 +36,26 @@
<properties>
<pinot.root>${basedir}/../../..</pinot.root>
<phase.prop>package</phase.prop>
- <aws.version>2.15.50</aws.version>
+ <aws.version>2.14.28</aws.version>
+ <powermock.version>2.0.9</powermock.version>
Review comment:
remove the powermock if it is not being used anymore
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #6410: Add unit tests in Kinesis consumer
Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #6410:
URL: https://github.com/apache/incubator-pinot/pull/6410#discussion_r567995099
##########
File path: pinot-plugins/pinot-stream-ingestion/pinot-kinesis/pom.xml
##########
@@ -36,9 +36,26 @@
<properties>
<pinot.root>${basedir}/../../..</pinot.root>
<phase.prop>package</phase.prop>
- <aws.version>2.15.50</aws.version>
+ <aws.version>2.14.28</aws.version>
+ <powermock.version>2.0.9</powermock.version>
Review comment:
resolved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] KKcorps commented on a change in pull request #6410: Add unit tests in Kinesis consumer
Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #6410:
URL: https://github.com/apache/incubator-pinot/pull/6410#discussion_r567994460
##########
File path: pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProvider.java
##########
@@ -83,7 +91,7 @@ public long fetchPartitionOffset(@Nonnull OffsetCriteria offsetCriteria, long ti
List<PartitionGroupInfo> newPartitionGroupInfos = new ArrayList<>();
Map<String, Shard> shardIdToShardMap =
- _kinesisConnectionHandler.getShards().stream().collect(Collectors.toMap(Shard::shardId, s -> s));
+ _kinesisConnectionHandler.getShards().stream().collect(Collectors.toMap(Shard::shardId, s -> s, (s1, s2) -> s1));
Review comment:
I also wasn't able to figure this out but while running a collision was definitely happening and error was being thrown.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] npawar merged pull request #6410: Add unit tests in Kinesis consumer
Posted by GitBox <gi...@apache.org>.
npawar merged pull request #6410:
URL: https://github.com/apache/incubator-pinot/pull/6410
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org