You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by QiangCai <gi...@git.apache.org> on 2017/04/13 14:02:33 UTC
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923]fix issue of insertI...
GitHub user QiangCai opened a pull request:
https://github.com/apache/incubator-carbondata/pull/795
[CARBONDATA-923]fix issue of insertInto read from OneRowRelation
Reproduce:
create table OneRowTable(col1 string, col2 string, col3 int, col4 double) stored by 'carbondata'
insert into OneRowTable select '0.1', 'a.b', 1, 1.2
Exception:
org.apache.spark.sql.AnalysisException: cannot resolve '`0.1`' given input columns: [0.1, a.b, 1, 1.2];;
'Project ['0.1, 'a.b]
+- Project 0.1 AS 0.1#11, a.b AS a.b#12, 1 AS 1#13, 1.2 AS 1.2#14
+- OneRowRelation$
Solution:
rename output attribute of select sub-query
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/QiangCai/incubator-carbondata fixInsertInto
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/795.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #795
----
commit 7c2bb5f970155a854f2b807e8aaf32de8bca43d8
Author: QiangCai <qi...@qq.com>
Date: 2017-04-13T13:09:12Z
insertInto read from OneRowRelation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923] Fix issue of insert...
Posted by QiangCai <gi...@git.apache.org>.
Github user QiangCai commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/795#discussion_r113606343
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
@@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
)
}
if (child.output.size >= relation.carbonRelation.output.size) {
- InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ var index = 0
+ val newChildOutput = child.output.map { column =>
+ index = index + 1
+ column match {
+ case attr: Alias =>
+ Alias(attr.child, s"col$index")(attr.exprId)
+ case attr: Attribute =>
+ Alias(attr, s"col$index")(NamedExpression.newExprId)
+ case attr => attr
+ }
+ }
+ if (newChildOutput == child.output) {
+ InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ } else {
+ InsertIntoCarbonTable(relation,
+ p.partition,
+ Project(newChildOutput, child),
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923] Fix issue of insert...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/795
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923] Fix issue of insert...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/795#discussion_r113556260
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
@@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
)
}
if (child.output.size >= relation.carbonRelation.output.size) {
- InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ var index = 0
+ val newChildOutput = child.output.map { column =>
--- End diff --
use `mapWithIndex` instead
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923]fix issue of insertInto rea...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1676/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923] Fix issue of insert...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/795#discussion_r113556346
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
@@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
)
}
if (child.output.size >= relation.carbonRelation.output.size) {
- InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ var index = 0
+ val newChildOutput = child.output.map { column =>
+ index = index + 1
+ column match {
+ case attr: Alias =>
+ Alias(attr.child, s"col$index")(attr.exprId)
+ case attr: Attribute =>
+ Alias(attr, s"col$index")(NamedExpression.newExprId)
+ case attr => attr
+ }
+ }
+ if (newChildOutput == child.output) {
+ InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ } else {
+ InsertIntoCarbonTable(relation,
--- End diff --
move `relation` to next line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923] Fix issue of insertInto re...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1814/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923]fix issue of insertInto rea...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1677/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923] Fix issue of insert...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/795#discussion_r113556768
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
@@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
)
}
if (child.output.size >= relation.carbonRelation.output.size) {
- InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ var index = 0
+ val newChildOutput = child.output.map { column =>
+ index = index + 1
+ column match {
+ case attr: Alias =>
+ Alias(attr.child, s"col$index")(attr.exprId)
+ case attr: Attribute =>
+ Alias(attr, s"col$index")(NamedExpression.newExprId)
+ case attr => attr
+ }
+ }
+ if (newChildOutput == child.output) {
+ InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ } else {
+ InsertIntoCarbonTable(relation,
+ p.partition,
+ Project(newChildOutput, child),
--- End diff --
I think you can move the creation of `InsertIntoCarbonTable` below else clause, so that it will take care of both if and else case. Only one parameter is different.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923] Fix issue of insert...
Posted by QiangCai <gi...@git.apache.org>.
Github user QiangCai commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/795#discussion_r113606332
--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
@@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
)
}
if (child.output.size >= relation.carbonRelation.output.size) {
- InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
+ var index = 0
+ val newChildOutput = child.output.map { column =>
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923]fix issue of insertInto rea...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1610/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923] Fix issue of insertInto re...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923]fix issue of insertInto rea...
Posted by QiangCai <gi...@git.apache.org>.
Github user QiangCai commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
@gvramana
I moved test case to spark 2.1. And spark 1.6 doesn't support it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923]fix issue of insertInto rea...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1638/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #795: [CARBONDATA-923] Fix issue of insertInto re...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/795
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1815/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---