You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by anubhav100 <gi...@git.apache.org> on 2018/05/09 11:21:36 UTC
[GitHub] carbondata pull request #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto ...
GitHub user anubhav100 opened a pull request:
https://github.com/apache/carbondata/pull/2287
[CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Query CarbonData When CarbonStore is at S3
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/anubhav100/incubator-carbondata presto/s3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2287.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2287
----
commit 9706f59df0f27769c6d3416d2daa4bb2d572e134
Author: anubhav100 <an...@...>
Date: 2018-05-01T04:29:05Z
Fixed Presto Can't Query CarbonData When CarbonStore is at s3
----
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4764/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4771/
---
[GitHub] carbondata pull request #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto ...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2287#discussion_r190794987
--- Diff: integration/presto/README.md ---
@@ -82,6 +82,29 @@ Please follow the below steps to query carbondata in presto
For example, if you have a schema named 'default' stored in hdfs://namenode:9000/test/carbondata/,
Then set carbondata-store=hdfs://namenode:9000/test/carbondata
+#### Connecting to carbondata store on s3
+ * In case you want to query carbonstore on S3 using S3A api put following additional properties inside $PRESTO_HOME$/etc/catalog/carbondata.properties
+ ```
+ fs.s3a.access.key={value}
+ fs.s3a.secret.key={value}
+ Optional: fs.s3a.endpoint={value}
--- End diff --
done
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on the issue:
https://github.com/apache/carbondata/pull/2287
retest this please
---
[GitHub] carbondata pull request #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto ...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2287#discussion_r191330878
--- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableConfig.java ---
@@ -60,6 +69,7 @@
return this;
}
+
--- End diff --
remove empty line
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4608/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6182/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2287
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4949/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on the issue:
https://github.com/apache/carbondata/pull/2287
@xuchuanyin @jackylk please review
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on the issue:
https://github.com/apache/carbondata/pull/2287
@jackylk i have done changes please review
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5926/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5021/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2287
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4826/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by anubhav100 <gi...@git.apache.org>.
Github user anubhav100 commented on the issue:
https://github.com/apache/carbondata/pull/2287
retest this please
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5918/
---
[GitHub] carbondata pull request #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2287
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2287
LGTM
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2287
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5156/
---
[GitHub] carbondata pull request #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto ...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2287#discussion_r191330841
--- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableConfig.java ---
@@ -51,7 +59,8 @@
return this;
}
- @NotNull public String getStorePath() {
+ @NotNull public String
+ getStorePath() {
--- End diff --
no need to change
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2287
please remove datamap.mdtfile file in the commit
@anubhav100
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5767/
---
[GitHub] carbondata issue #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto Can't Q...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2287
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4616/
---
[GitHub] carbondata pull request #2287: [CARBONDATA-2418] [Presto] [S3] Fixed Presto ...
Posted by xuchuanyin <gi...@git.apache.org>.
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2287#discussion_r187768917
--- Diff: integration/presto/README.md ---
@@ -82,6 +82,29 @@ Please follow the below steps to query carbondata in presto
For example, if you have a schema named 'default' stored in hdfs://namenode:9000/test/carbondata/,
Then set carbondata-store=hdfs://namenode:9000/test/carbondata
+#### Connecting to carbondata store on s3
+ * In case you want to query carbonstore on S3 using S3A api put following additional properties inside $PRESTO_HOME$/etc/catalog/carbondata.properties
+ ```
+ fs.s3a.access.key={value}
+ fs.s3a.secret.key={value}
+ Optional: fs.s3a.endpoint={value}
--- End diff --
better to describe the properties as below to make it convenient for user to copy&paste.
```
# Required properties
A=B
# Optional properties
C=D
```
---