You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Aravindh <ma...@aravindh.io> on 2016/09/20 05:49:16 UTC

Configuring Kinesis max records limit in KinesisReceiver

I use `KinesisUtil.createStream` to create a DStream from a kinesis stream.
By default my spark receiver receives 10000 events from the stream. I see
that the default value for KCL comes from  KCL Configuration
<https://github.com/awslabs/amazon-kinesis-client/blob/master/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/KinesisClientLibConfiguration.java#L52> 
. How do I override this? I dont see a way to pass configuration to 
`KinesisUtil.createStream` for overriding the KCL configuration.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Configuring-Kinesis-max-records-limit-in-KinesisReceiver-tp27763.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org