You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Brian London (JIRA)" <ji...@apache.org> on 2015/12/03 20:27:10 UTC
[jira] [Updated] (SPARK-12134) Spark Streaming Kinesis: Allow a
mechanism to bypass validateRegion
[ https://issues.apache.org/jira/browse/SPARK-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian London updated SPARK-12134:
---------------------------------
Description:
The {{KinesisUtils.createStream}} family of methods internally call a function {{validateRegion}} that requires the supplied region to be one of the strings on the list of AWS regions. Doing so, prevents the use of createStream to connect to a non-AWS kinesis stream such as one run through [Kinesalite|https://github.com/mhart/kinesalite], which can be useful for testing. The {{validateRegion}} method does no additional transformation so it's only effective function is to kill the createStream call if a user requests a stream with a non-standard region string (intentionally or otherwise).
There should be a mechanism to bypass this check, if not remove it entirely.
was:
The KinesisUtils.createStream family of methods internally call a function validateRegion that requires the supplied region to be one of the strings on the list of AWS regions. Doing so, prevents the use of createStream to connect to a non-AWS kinesis stream such as one run through Kinesalite, which can be useful for testing. The getRegion method does no additional transformation so it's only effective function is to kill the createStream call if a user requests a stream with a non-standard region string (intentionally or otherwise).
There should be a mechanism to bypass this check, if not remove it entirely.
> Spark Streaming Kinesis: Allow a mechanism to bypass validateRegion
> -------------------------------------------------------------------
>
> Key: SPARK-12134
> URL: https://issues.apache.org/jira/browse/SPARK-12134
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.5.2
> Reporter: Brian London
>
> The {{KinesisUtils.createStream}} family of methods internally call a function {{validateRegion}} that requires the supplied region to be one of the strings on the list of AWS regions. Doing so, prevents the use of createStream to connect to a non-AWS kinesis stream such as one run through [Kinesalite|https://github.com/mhart/kinesalite], which can be useful for testing. The {{validateRegion}} method does no additional transformation so it's only effective function is to kill the createStream call if a user requests a stream with a non-standard region string (intentionally or otherwise).
> There should be a mechanism to bypass this check, if not remove it entirely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org