You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by ajantha-bhat <gi...@git.apache.org> on 2018/05/29 06:37:35 UTC
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2352
[CARBONDATA-2555]Fixed SDK reader set default isTransactional as false
[CARBONDATA-2555]Fixed SDK reader set default isTransactional as false .
SDK writer is having default value of isTransactional is false. But reader is not like this.
So, Fixing this by deafult make SDK to use flat folder structure.
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed? yes. Changed the default value
- [ ] Any backward compatibility impacted? NA
- [ ] Document update required? yes, updated
- [ ] Testing done
Updated the UT
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajantha-bhat/carbondata issue_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2352.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2352
----
commit 869930514cc7fd37c02ec3e8c8e2d1ba2063c814
Author: ajantha-bhat <aj...@...>
Date: 2018-05-29T06:33:55Z
[CARBONDATA-2555] SDK reader set default isTransactional as false
----
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5013/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/2352
please rebase
---
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2352#discussion_r191691253
--- Diff: store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -305,7 +309,7 @@ public void testWriteAndReadFilesNonTransactional() throws IOException, Interrup
// Write to a Non Transactional Table
TestUtil.writeFilesAndVerify(new Schema(fields), path, true, false);
- CarbonReader reader = CarbonReader.builder(path, "_temp")
+ CarbonReader reader = CarbonReader.builder(path, "_temp").isTransactionalTable(true)
--- End diff --
This is a conflict resolve problem. Will be removed in the next commit. Finally it should be false for this testcase
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6163/
---
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2352#discussion_r191695689
--- Diff: store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -305,7 +309,7 @@ public void testWriteAndReadFilesNonTransactional() throws IOException, Interrup
// Write to a Non Transactional Table
TestUtil.writeFilesAndVerify(new Schema(fields), path, true, false);
- CarbonReader reader = CarbonReader.builder(path, "_temp")
+ CarbonReader reader = CarbonReader.builder(path, "_temp").isTransactionalTable(true)
--- End diff --
ok, It's fine.
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/2352
retest this please
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4991/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6174/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2352
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5134/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2352
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5149/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2352
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5135/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6153/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6171/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by sounakr <gi...@git.apache.org>.
Github user sounakr commented on the issue:
https://github.com/apache/carbondata/pull/2352
LGTM
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5003/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/2352
LGTM
---
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2352#discussion_r191690950
--- Diff: store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -305,7 +309,7 @@ public void testWriteAndReadFilesNonTransactional() throws IOException, Interrup
// Write to a Non Transactional Table
TestUtil.writeFilesAndVerify(new Schema(fields), path, true, false);
- CarbonReader reader = CarbonReader.builder(path, "_temp")
+ CarbonReader reader = CarbonReader.builder(path, "_temp").isTransactionalTable(true)
--- End diff --
This came by conflict resolve. This will be removed in the next commit. No harm.
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2352
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5147/
---
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2352
---
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2352#discussion_r191695582
--- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/SDKS3Example.java ---
@@ -44,34 +44,14 @@ public static void main(String[] args) throws Exception {
num = Integer.parseInt(args[4]);
}
- Boolean persistSchema = true;
- if (args.length > 5) {
- if (args[5].equalsIgnoreCase("true")) {
- persistSchema = true;
- } else {
- persistSchema = false;
- }
- }
-
- Boolean transactionalTable = true;
- if (args.length > 6) {
- if (args[6].equalsIgnoreCase("true")) {
- transactionalTable = true;
- } else {
- transactionalTable = false;
- }
- }
-
Field[] fields = new Field[2];
fields[0] = new Field("name", DataTypes.STRING);
fields[1] = new Field("age", DataTypes.INT);
CarbonWriterBuilder builder = CarbonWriter.builder()
.setAccessKey(args[0])
.setSecretKey(args[1])
.setEndPoint(args[2])
- .outputPath(path)
- .persistSchemaFile(persistSchema)
- .isTransactionalTable(transactionalTable);
--- End diff --
remove this one , this example can't change value of isTransactionalTable and persistSchemaFile when we test it. I suggest it better to keep it.
---
[GitHub] carbondata pull request #2352: [CARBONDATA-2555]Fixed SDK reader set default...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2352#discussion_r191690080
--- Diff: store/sdk/src/test/java/org/apache/carbondata/sdk/file/CarbonReaderTest.java ---
@@ -305,7 +309,7 @@ public void testWriteAndReadFilesNonTransactional() throws IOException, Interrup
// Write to a Non Transactional Table
TestUtil.writeFilesAndVerify(new Schema(fields), path, true, false);
- CarbonReader reader = CarbonReader.builder(path, "_temp")
+ CarbonReader reader = CarbonReader.builder(path, "_temp").isTransactionalTable(true)
--- End diff --
There has invoked isTransactionalTable method in line 314, why invoke again?
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5010/
---
[GitHub] carbondata issue #2352: [CARBONDATA-2555]Fixed SDK reader set default isTran...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2352
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6151/
---