You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by Indhumathi27 <gi...@git.apache.org> on 2018/07/30 08:53:35 UTC
[GitHub] carbondata pull request #2578: Fix Dictionary_Include for ComplexDataType
GitHub user Indhumathi27 opened a pull request:
https://github.com/apache/carbondata/pull/2578
Fix Dictionary_Include for ComplexDataType
Why this PR for?
Check if a column is given as Dictionary_Include and fill cardinality for that column
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Indhumathi27/carbondata dict_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2578.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2578
----
commit bab79be62af7ffecb9dc1fcfe520fd0288c0281a
Author: Indhumathi27 <in...@...>
Date: 2018-07-30T08:48:44Z
Fix Dictionary_Include for ComplexDataType
----
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by brijoobopanna <gi...@git.apache.org>.
Github user brijoobopanna commented on the issue:
https://github.com/apache/carbondata/pull/2578
retest sdv please
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6378/
---
[GitHub] carbondata pull request #2578: [CARBONDATA-2798] Fix Dictionary_Include for ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2578
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by brijoobopanna <gi...@git.apache.org>.
Github user brijoobopanna commented on the issue:
https://github.com/apache/carbondata/pull/2578
retest this please
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/2578
Please add a test case to verify the fix
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7628/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7658/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by Indhumathi27 <gi...@git.apache.org>.
Github user Indhumathi27 commented on the issue:
https://github.com/apache/carbondata/pull/2578
Retest this please
---
[GitHub] carbondata pull request #2578: [CARBONDATA-2798] Fix Dictionary_Include for ...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2578#discussion_r206531160
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/createTable/TestNonTransactionalCarbonTable.scala ---
@@ -52,10 +52,11 @@ import org.apache.carbondata.core.datastore.page.encoding.DefaultEncodingFactory
import org.apache.carbondata.core.metadata.ColumnarFormatVersion
import org.apache.carbondata.core.metadata.datatype.DataTypes
import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil, DataFileFooterConverterV3}
+import org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException
import org.apache.carbondata.sdk.file._
-class TestNonTransactionalCarbonTable extends QueryTest with BeforeAndAfterAll {
+class TestNonTransactiogitnalCarbonTable extends QueryTest with BeforeAndAfterAll {
--- End diff --
Please revert this change
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7638/
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7672/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7607/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2578
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6071/
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6392/
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7684/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7615/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6358/
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2578
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6085/
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/2578
LGTM
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6402/
---
[GitHub] carbondata issue #2578: Fix Dictionary_Include for ComplexDataType
Posted by brijoobopanna <gi...@git.apache.org>.
Github user brijoobopanna commented on the issue:
https://github.com/apache/carbondata/pull/2578
retest this please
---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2578
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6408/
---
[GitHub] carbondata pull request #2578: Fix Dictionary_Include for ComplexDataType
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2578#discussion_r206091924
--- Diff: processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java ---
@@ -172,7 +191,7 @@ public void setSurrogateIndex(int surrIndex) {
@Override
public void fillCardinality(List<Integer> dimCardWithComplex) {
- if (children.getIsColumnDictionary()) {
+ if (this.isDictionaryColumn) {
--- End diff --
return isDictionaryColumn from getIsColumnDictionary() also.
---
[GitHub] carbondata pull request #2578: Fix Dictionary_Include for ComplexDataType
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2578#discussion_r206396378
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/model/LoadOption.java ---
@@ -236,7 +236,7 @@
}
}
- if (carbonLoadModel.isCarbonTransactionalTable() && !CarbonDataProcessorUtil
+ if (hadoopConf != null && !CarbonDataProcessorUtil
--- End diff --
why hadoopConf null check is needed?
---