You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by ajantha-bhat <gi...@git.apache.org> on 2018/04/06 08:43:38 UTC
[GitHub] carbondata pull request #2141: Added comment for SDK writer API methods
GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2141
Added comment for SDK writer API methods
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed? NO
- [ ] Any backward compatibility impacted? NO
- [ ] Document update required? NO
- [ ] Testing done. NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajantha-bhat/carbondata unmanaged_table
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2141.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2141
----
commit 49970829d56b91f2a5ca6e46f3957b248cf9ccde
Author: ajantha-bhat <aj...@...>
Date: 2018-04-06T08:40:38Z
Added comment for SDK writer API methods
----
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3844/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3667/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
Retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5052/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4346/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
Retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Added comment for SDK writer API m...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4320/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3704/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181632750
--- Diff: core/src/main/java/org/apache/carbondata/core/scan/executor/util/RestructureUtil.java ---
@@ -354,7 +369,13 @@ public static Object getMeasureDefaultValueByType(ColumnSchema columnSchema,
// then setting measure exists is true
// otherwise adding a default value of a measure
for (CarbonMeasure carbonMeasure : currentBlockMeasures) {
- if (carbonMeasure.getColumnId().equals(queryMeasure.getMeasure().getColumnId())) {
+ // If it is unmanaged table just check the column names, no need to validate column id as
+ // multiple sdk's output placed in a single folder doesn't have same column ID but can
+ // have same column name
+ if (carbonMeasure.getColumnId().equals(queryMeasure.getMeasure().getColumnId()) ||
+ ((queryModel != null) && (queryModel.getTable().getTableInfo().isUnManagedTable()) &&
--- End diff --
remove null check and add method isColumnMatches
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Added comment for SDK writer API m...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4868/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4890/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4439/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181646642
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala ---
@@ -73,22 +73,27 @@ private[sql] case class CarbonAlterTableRenameCommand(
s"Table $oldDatabaseName.$oldTableName does not exist")
throwMetadataException(oldDatabaseName, oldTableName, "Table does not exist")
}
+
+ var carbonTable: CarbonTable = null
+ carbonTable = metastore.lookupRelation(Some(oldDatabaseName), oldTableName)(sparkSession)
--- End diff --
OK. modified
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2141
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4345/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4925/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4990/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5077/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181650233
--- Diff: core/src/main/java/org/apache/carbondata/core/scan/executor/util/RestructureUtil.java ---
@@ -354,7 +369,13 @@ public static Object getMeasureDefaultValueByType(ColumnSchema columnSchema,
// then setting measure exists is true
// otherwise adding a default value of a measure
for (CarbonMeasure carbonMeasure : currentBlockMeasures) {
- if (carbonMeasure.getColumnId().equals(queryMeasure.getMeasure().getColumnId())) {
+ // If it is unmanaged table just check the column names, no need to validate column id as
+ // multiple sdk's output placed in a single folder doesn't have same column ID but can
+ // have same column name
+ if (carbonMeasure.getColumnId().equals(queryMeasure.getMeasure().getColumnId()) ||
+ ((queryModel != null) && (queryModel.getTable().getTableInfo().isUnManagedTable()) &&
--- End diff --
removed null check. cannot call from a method as one is for measure type and other is a dimension type.
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181632323
--- Diff: core/src/main/java/org/apache/carbondata/core/scan/executor/util/RestructureUtil.java ---
@@ -337,11 +351,12 @@ public static Object getMeasureDefaultValueByType(ColumnSchema columnSchema,
* @param blockExecutionInfo
* @param queryMeasures measures present in query
* @param currentBlockMeasures current block measures
+ * @param queryModel carbonQueryModel
* @return measures present in the block
*/
public static List<ProjectionMeasure> createMeasureInfoAndGetCurrentBlockQueryMeasures(
BlockExecutionInfo blockExecutionInfo, List<ProjectionMeasure> queryMeasures,
- List<CarbonMeasure> currentBlockMeasures) {
+ List<CarbonMeasure> currentBlockMeasures, QueryModel queryModel) {
--- End diff --
Pass flag, dont pass all the querymodel, Restructuring doesnot have any relation with Querymodel
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3778/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181648541
--- Diff: core/src/main/java/org/apache/carbondata/core/scan/executor/util/RestructureUtil.java ---
@@ -337,11 +351,12 @@ public static Object getMeasureDefaultValueByType(ColumnSchema columnSchema,
* @param blockExecutionInfo
* @param queryMeasures measures present in query
* @param currentBlockMeasures current block measures
+ * @param queryModel carbonQueryModel
* @return measures present in the block
*/
public static List<ProjectionMeasure> createMeasureInfoAndGetCurrentBlockQueryMeasures(
BlockExecutionInfo blockExecutionInfo, List<ProjectionMeasure> queryMeasures,
- List<CarbonMeasure> currentBlockMeasures) {
+ List<CarbonMeasure> currentBlockMeasures, QueryModel queryModel) {
--- End diff --
OK
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3818/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4952/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3771/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3856/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3788/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3654/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4344/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5080/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181634024
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/createTable/TestUnmanagedCarbonTable.scala ---
@@ -266,16 +270,23 @@ class TestUnmanagedCarbonTable extends QueryTest with BeforeAndAfterAll {
.contains("Unsupported operation on unmanaged table"))
//12. Streaming table creation
- // External table don't accept table properties
+ // No need as External table don't accept table properties
+
+ //13. Alter table rename command
+ exception = intercept[MalformedCarbonCommandException] {
+ sql("ALTER TABLE sdkOutputTable RENAME to newTable")
+ }
+ assert(exception.getMessage()
+ .contains("Unsupported operation on unmanaged table"))
sql("DROP TABLE sdkOutputTable")
- // drop table should not delete the files
+ //drop table should not delete the files
--- End diff --
correct indent
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181646540
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala ---
@@ -565,7 +563,9 @@ class CarbonFileMetastore extends CarbonMetaStore {
tableModifiedTimeStore.get(CarbonCommonConstants.DATABASE_DEFAULT_NAME))) {
metadata.carbonTables = metadata.carbonTables.filterNot(
table => table.getTableName.equalsIgnoreCase(tableIdentifier.table) &&
- table.getDatabaseName.equalsIgnoreCase(tableIdentifier.database.getOrElse("default")))
+ table.getDatabaseName.equalsIgnoreCase(
--- End diff --
Ran all the test case. It is not coming now. Removed this changes
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5004/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Added comment for SDK writer API m...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4337/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Added comment for SDK write...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r179951717
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java ---
@@ -58,44 +58,85 @@
private boolean isUnManagedTable;
private long UUID;
+ /**
+ * prepares the builder with the schema provided
+ * @param schema
--- End diff --
done. Added.
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5049/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181637617
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala ---
@@ -565,7 +563,9 @@ class CarbonFileMetastore extends CarbonMetaStore {
tableModifiedTimeStore.get(CarbonCommonConstants.DATABASE_DEFAULT_NAME))) {
metadata.carbonTables = metadata.carbonTables.filterNot(
table => table.getTableName.equalsIgnoreCase(tableIdentifier.table) &&
- table.getDatabaseName.equalsIgnoreCase(tableIdentifier.database.getOrElse("default")))
+ table.getDatabaseName.equalsIgnoreCase(
--- End diff --
Why it coming here for unmanged table
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Added comment for SDK writer API m...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4851/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4388/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4995/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
Retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4435/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Added comment for SDK write...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r179950388
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java ---
@@ -58,44 +58,85 @@
private boolean isUnManagedTable;
private long UUID;
+ /**
+ * prepares the builder with the schema provided
+ * @param schema
--- End diff --
provide description for @param and @return
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
@jackylk & @gvramana : please review this PR.
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4877/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181634921
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala ---
@@ -73,22 +73,27 @@ private[sql] case class CarbonAlterTableRenameCommand(
s"Table $oldDatabaseName.$oldTableName does not exist")
throwMetadataException(oldDatabaseName, oldTableName, "Table does not exist")
}
+
+ var carbonTable: CarbonTable = null
+ carbonTable = metastore.lookupRelation(Some(oldDatabaseName), oldTableName)(sparkSession)
--- End diff --
This cannot be moved outside lock
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5061/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3835/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on the issue:
https://github.com/apache/carbondata/pull/2141
LGTM
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2141
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4409/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed drop table issue in cluster ...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2141
retest this please
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181648562
--- Diff: core/src/main/java/org/apache/carbondata/core/scan/executor/util/RestructureUtil.java ---
@@ -354,7 +369,13 @@ public static Object getMeasureDefaultValueByType(ColumnSchema columnSchema,
// then setting measure exists is true
// otherwise adding a default value of a measure
for (CarbonMeasure carbonMeasure : currentBlockMeasures) {
- if (carbonMeasure.getColumnId().equals(queryMeasure.getMeasure().getColumnId())) {
+ // If it is unmanaged table just check the column names, no need to validate column id as
+ // multiple sdk's output placed in a single folder doesn't have same column ID but can
+ // have same column name
+ if (carbonMeasure.getColumnId().equals(queryMeasure.getMeasure().getColumnId()) ||
+ ((queryModel != null) && (queryModel.getTable().getTableInfo().isUnManagedTable()) &&
--- End diff --
ok
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5034/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Added comment for SDK writer API m...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3629/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5053/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3710/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5051/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3839/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4929/
---
[GitHub] carbondata pull request #2141: [CARBONDATA-2313] Fixed SDK writer issues and...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2141#discussion_r181647929
--- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/createTable/TestUnmanagedCarbonTable.scala ---
@@ -266,16 +270,23 @@ class TestUnmanagedCarbonTable extends QueryTest with BeforeAndAfterAll {
.contains("Unsupported operation on unmanaged table"))
//12. Streaming table creation
- // External table don't accept table properties
+ // No need as External table don't accept table properties
+
+ //13. Alter table rename command
+ exception = intercept[MalformedCarbonCommandException] {
+ sql("ALTER TABLE sdkOutputTable RENAME to newTable")
+ }
+ assert(exception.getMessage()
+ .contains("Unsupported operation on unmanaged table"))
sql("DROP TABLE sdkOutputTable")
- // drop table should not delete the files
+ //drop table should not delete the files
--- End diff --
done
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5066/
---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Added comment for SDK writer API m...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2141
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3646/
---