You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by ravipesala <gi...@git.apache.org> on 2016/10/07 17:31:41 UTC
[GitHub] incubator-carbondata pull request #215: [WIP][CARBONDATA-2] Remove kettle fr...
GitHub user ravipesala opened a pull request:
https://github.com/apache/incubator-carbondata/pull/215
[WIP][CARBONDATA-2] Remove kettle from carbodata
New flow is added in processing module to load data with out kettle. New code is added to the package `org.apache.carbondata.processing.newflow` , old code (kettle framework) will be removed once all performance comparisons are met.
This is work in progress PR and not in working condition. All interfaces and related code is added but need to be tested.
Main interfaces for loading the data.
`org.apache.carbondata.processing.newflow.DataLoadProcessorStep`
`org.apache.carbondata.processing.newflow.dictionarygenerator.ColumnDictionaryGenerator`
`org.apache.carbondata.processing.newflow.encoding.RowEncoder`
`org.apache.carbondata.processing.newflow.parser.GenericParser`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ravipesala/incubator-carbondata remove-kettle
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/215.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #215
----
commit 376933cefe5f2f241af460829373d7e9b381322d
Author: ravipesala <ra...@gmail.com>
Date: 2016-06-05T17:08:22Z
WIP interfaces created
commit abc6f489cfb84e5697adff3494b34873dc70a434
Author: ravipesala <ra...@gmail.com>
Date: 2016-09-08T19:14:39Z
Added interfaces for data loading
commit 0922915035b1c016f1b82556025d34846ee598c4
Author: ravipesala <ra...@gmail.com>
Date: 2016-09-20T10:28:54Z
[WIP]Added interfaces for data loading
commit 1f1500adbb7097bc69eaa2238da0939933c298b3
Author: ravipesala <ra...@gmail.com>
Date: 2016-10-04T14:34:58Z
Updated package and added sort step
commit 5625bf9433488d5acd4b37e30dbc8751dc8a0e1e
Author: ravipesala <ra...@gmail.com>
Date: 2016-10-07T09:04:28Z
Added writer step
commit 99fd9f7f117b8221a0fced62678da35239a7d2a0
Author: ravipesala <ra...@gmail.com>
Date: 2016-10-07T17:14:23Z
Added comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #215: [WIP][CARBONDATA-2] Remove kettle fr...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/215#discussion_r82504432
--- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/csvinput/CustomArrayWritable.java ---
@@ -0,0 +1,51 @@
+package org.apache.carbondata.hadoop.csvinput;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.util.Arrays;
+
+import org.apache.hadoop.io.Writable;
+
+/**
+ * Created by root1 on 16/4/16.
--- End diff --
please remove
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #215: [WIP][CARBONDATA-2] Remove kettle fr...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/215
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---