[GitHub] [druid] cryptoe commented on a change in pull request #12161: Mitigate Kinesis stream LimitExceededException by using listShards API

[GitBox <gi...@apache.org>]

cryptoe commented on a change in pull request #12161:
URL: https://github.com/apache/druid/pull/12161#discussion_r785663991



##########
File path: integration-tests/src/main/java/org/apache/druid/testing/utils/KinesisAdminClient.java
##########
@@ -156,15 +162,16 @@ public boolean verfiyPartitionCountUpdated(String streamName, int oldShardCount,
     return actualShardCount == oldShardCount + newShardCount;
   }
 
-
-  private boolean verifyStreamStatus(StreamDescription streamDescription, StreamStatus streamStatusToCheck)
+  private List<Shard> listShards(String streamName)
   {
-    return streamStatusToCheck.toString().equals(streamDescription.getStreamStatus());

Review comment:
       Can we add a comment with the aws tps limits currently for the `listshards` call ? 

##########
File path: integration-tests/src/main/java/org/apache/druid/testing/utils/KinesisAdminClient.java
##########
@@ -107,6 +111,9 @@ public void deleteStream(String streamName)
   public void updatePartitionCount(String streamName, int newShardCount, boolean blocksUntilStarted)
   {
     int originalShardCount = getStreamPartitionCount(streamName);
+    if (originalShardCount == newShardCount) {

Review comment:
       Seems like a candidate for a test case if there are none for this change.

##########
File path: integration-tests/src/main/java/org/apache/druid/testing/utils/KinesisAdminClient.java
##########
@@ -30,15 +30,19 @@
 import com.amazonaws.services.kinesis.model.CreateStreamResult;
 import com.amazonaws.services.kinesis.model.DeleteStreamResult;
 import com.amazonaws.services.kinesis.model.DescribeStreamResult;
+import com.amazonaws.services.kinesis.model.ListShardsRequest;
 import com.amazonaws.services.kinesis.model.ScalingType;
+import com.amazonaws.services.kinesis.model.Shard;
 import com.amazonaws.services.kinesis.model.StreamDescription;
 import com.amazonaws.services.kinesis.model.StreamStatus;
 import com.amazonaws.services.kinesis.model.UpdateShardCountRequest;
 import com.amazonaws.services.kinesis.model.UpdateShardCountResult;
 import com.amazonaws.util.AwsHostNameUtils;
 import org.apache.druid.java.util.common.ISE;
+import org.apache.hadoop.yarn.webapp.hamlet.Hamlet;

Review comment:
       Looks like an accidental import.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org