You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Rustam Aliyev (JIRA)" <ji...@apache.org> on 2015/09/08 15:57:45 UTC
[jira] [Commented] (SPARK-6101) Create a SparkSQL DataSource API
implementation for DynamoDB
[ https://issues.apache.org/jira/browse/SPARK-6101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734830#comment-14734830 ]
Rustam Aliyev commented on SPARK-6101:
--------------------------------------
What's the status of this? GH repo has not been updated for a while.
Few improvements which I'd like to see:
# Use {FilterExpression} instead of legacy {ScanFilter} in the {scan}
# Leverage Parallel Scan (http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html#QueryAndScanParallelScan) and make multiple Spark workers pull data in parallel.
> Create a SparkSQL DataSource API implementation for DynamoDB
> ------------------------------------------------------------
>
> Key: SPARK-6101
> URL: https://issues.apache.org/jira/browse/SPARK-6101
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.2.0
> Reporter: Chris Fregly
> Assignee: Chris Fregly
> Fix For: 1.6.0
>
>
> similar to https://github.com/databricks/spark-avro and https://github.com/databricks/spark-csv
> Here's a good basis for a java-based, high-level dynamodb java connector: https://github.com/sporcina/dynamodb-connector/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org