You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:43:12 UTC
[jira] [Resolved] (SPARK-24218) Allow Configuration of
DynamoDbEndpointUrl in KinesisReceiver
[ https://issues.apache.org/jira/browse/SPARK-24218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-24218.
----------------------------------
Resolution: Incomplete
> Allow Configuration of DynamoDbEndpointUrl in KinesisReceiver
> -------------------------------------------------------------
>
> Key: SPARK-24218
> URL: https://issues.apache.org/jira/browse/SPARK-24218
> Project: Spark
> Issue Type: Improvement
> Components: DStreams
> Affects Versions: 2.3.0
> Reporter: Cole Murray
> Priority: Major
> Labels: bulk-closed
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> KinesisReceiver does not allow users to set DynamoDbEndpointUrl. When testing Kinesis locally, users are unable to utilize local checkpoints through a local Dynamodb.
>
> An update to the constructor would allow other Apis, specifically, KinesisInputDStream.builder to add support for local DynamoDb endpointUrls.
>
> Resources:
> [https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala|KinesisReceiver]
> [https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala#L162|Line To Add EndpointUrl to KinesisClientLibConfiguration]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org