[jira] [Commented] (FLINK-9266) Upgrade AWS Kinesis Client version to 1.9.0 to reduce Kinesis describe streams calls

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456902#comment-16456902 ] 

ASF GitHub Bot commented on FLINK-9266:
---------------------------------------

GitHub user tmoser2525 opened a pull request:

    https://github.com/apache/flink/pull/5932

    [FLINK-9266][flink-connector-kinesis]Updates Kinesis connector to use newer version of kcl to limit describe streams calls

    ## What is the purpose of the change
    
    Updates Kinesis Connector to use newest version of AWS Kinesis Client Library (KCL). The new version of the KCL removes the usage of Kinesis Describe Stream calls to alleviate account wide limits.
    
    
    ## Brief change log
    
    *Flink-Kinesis-Connector*
      - *Upgrades KCL version to 1.9.0*
      - *Upgrades KPL version to 0.12.9*
      - *Upgrades AWS SDK version to 1.11.319*
    
    
    ## Verifying this change
    
    This change is a trivial rework / code cleanup without any test coverage.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (yes)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (don't know)
      - The S3 file system connector: (no)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tmoser2525/flink feature/upgrade-kinesis

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5932.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5932
    
----
commit 0e55eb6087934fbae65097705aeca58a0d4936d8
Author: Moser Thomas W <mo...@...>
Date:   2018-04-27T18:27:03Z

    [Flink-9266] Updates Kinesis connector to use newer version of kcl to limit describe streams calls

----


> Upgrade AWS Kinesis Client version to 1.9.0 to reduce Kinesis describe streams calls
> ------------------------------------------------------------------------------------
>
>                 Key: FLINK-9266
>                 URL: https://issues.apache.org/jira/browse/FLINK-9266
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kinesis Connector
>            Reporter: Thomas Moser
>            Priority: Trivial
>
> Versions of the AWS Kinesis Client before 1.9.0 can cause a large number of Kinesis Describe Stream events which can result in other AWS services such as AWS CloudFormation failing to deploy any stacks with Kinesis streams in them. For accounts that use a large number of Kinesis streams and Flink clusters this can cripple the ability to deploy anything in the account. Bumping the version of the KCL to 1.9.0 changes to a different underlying call that has a much larger AWS account limit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)