You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by kunal642 <gi...@git.apache.org> on 2018/05/17 15:34:26 UTC
[GitHub] carbondata pull request #2316: [WIP] Removed method to accept carbon schema ...
GitHub user kunal642 opened a pull request:
https://github.com/apache/carbondata/pull/2316
[WIP] Removed method to accept carbon schema in avro writer
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kunal642/carbondata avro_carbon_changes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2316.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2316
----
commit 0bec8317c2bda90267dd9ff4fee13f4d40333957
Author: kunal642 <ku...@...>
Date: 2018-05-17T15:30:50Z
removed method to accept carbon schema in avro writer
----
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4793/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5949/
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4836/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5950/
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4842/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4792/
---
[GitHub] carbondata pull request #2316: [CARBONDATA-2498] Change CarbonWriterBuilder ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2316
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4981/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5948/
---
[GitHub] carbondata pull request #2316: [WIP] Removed method to accept carbon schema ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2316#discussion_r189423162
--- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
@@ -44,11 +44,10 @@ public static void main(String[] args) {
fields[1] = new Field("age", DataTypes.INT);
CarbonWriter writer = CarbonWriter.builder()
- .withSchema(new Schema(fields))
--- End diff --
Is withSchema is optional earlier or mandatory to provide ?
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
LGTM
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5015/
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5996/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4791/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4978/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5947/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4794/
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6001/
---
[GitHub] carbondata pull request #2316: [CARBONDATA-2498] Change CarbonWriterBuilder ...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2316#discussion_r189530705
--- Diff: docs/sdk-writer-guide.md ---
@@ -33,9 +33,9 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
Schema schema = new Schema(fields);
- CarbonWriterBuilder builder = CarbonWriter.builder().withSchema(schema).outputPath(path);
--- End diff --
done
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4982/
---
[GitHub] carbondata pull request #2316: [CARBONDATA-2498] Change CarbonWriterBuilder ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2316#discussion_r189499110
--- Diff: docs/sdk-writer-guide.md ---
@@ -93,9 +93,8 @@ public class TestSdkAvro {
try {
CarbonWriter writer = CarbonWriter.builder()
- .withSchema(carbonSchema)
.outputPath(path)
- .buildWriterForAvroInput();
+ .buildWriterForAvroInput(new org.apache.avro.Schema.Parser().parse(avroSchema));
--- End diff --
remove these above lines. No need to prepare avro schema now
// prepare carbon schema from avro schema
org.apache.carbondata.sdk.file.Schema carbonSchema = org.apache.carbondata.sdk.file.Schema carbonSchema =
AvroCarbonWriter.getCarbonSchemaFromAvroSchema(avroSchema); AvroCarbonWriter.getCarbonSchemaFromAvroSchema(avroSchema);
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5991/
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4833/
---
[GitHub] carbondata issue #2316: [CARBONDATA-2498] Change CarbonWriterBuilder interfa...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5011/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4980/
---
[GitHub] carbondata pull request #2316: [CARBONDATA-2498] Change CarbonWriterBuilder ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2316#discussion_r189496847
--- Diff: docs/sdk-writer-guide.md ---
@@ -33,9 +33,9 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
Schema schema = new Schema(fields);
- CarbonWriterBuilder builder = CarbonWriter.builder().withSchema(schema).outputPath(path);
--- End diff --
In the same document handle the below points.
1. remove withSchema() interface
2. update the below interfaces,
buildWriterForCSVInput
buildWriterForAvroInput
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2316
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4979/
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4795/
---
[GitHub] carbondata pull request #2316: [WIP] Removed method to accept carbon schema ...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2316#discussion_r189495550
--- Diff: examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java ---
@@ -44,11 +44,10 @@ public static void main(String[] args) {
fields[1] = new Field("age", DataTypes.INT);
CarbonWriter writer = CarbonWriter.builder()
- .withSchema(new Schema(fields))
--- End diff --
mandatory to provide
---
[GitHub] carbondata pull request #2316: [CARBONDATA-2498] Change CarbonWriterBuilder ...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2316#discussion_r189530725
--- Diff: docs/sdk-writer-guide.md ---
@@ -93,9 +93,8 @@ public class TestSdkAvro {
try {
CarbonWriter writer = CarbonWriter.builder()
- .withSchema(carbonSchema)
.outputPath(path)
- .buildWriterForAvroInput();
+ .buildWriterForAvroInput(new org.apache.avro.Schema.Parser().parse(avroSchema));
--- End diff --
removed
---
[GitHub] carbondata issue #2316: [WIP] Removed method to accept carbon schema in avro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2316
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5951/
---