You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2020/03/02 04:21:27 UTC
[GitHub] [carbondata] nihal0107 opened a new pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
nihal0107 opened a new pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647
…sort column.
Why is this PR needed?
Currently the column name with different case(upper and lower)
is getting considered as different columns for sort properties.
What changes were proposed in this PR?
Added the case insensitive validation for duplicate sort column.
Does this PR introduce any user interface change?
No
Is any new testcase added?
No
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593839205
Build Failed with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/575/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593903819
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2284/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] asfgit closed pull request #3647: [CARBONDATA-3348]
Fix the case insensitive validation for duplicate sort column.
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593375823
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2266/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Zhangshunyu commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
Zhangshunyu commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593269015
the failed testcases need to change the message in assert which has changed with code modification
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3647: [CARBONDATA-3348]
Fix the case insensitive validation for duplicate sort column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593859090
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3647: [CARBONDATA-3348]
Fix the case insensitive validation for duplicate sort column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593914836
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3647: [CARBONDATA-3348]
Fix the case insensitive validation for duplicate sort column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593838402
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593867487
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/579/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593432763
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2267/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3647: [CARBONDATA-3348]
Fix the case insensitive validation for duplicate sort column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593838340
add to whitelist
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386812861
##########
File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##########
@@ -760,11 +760,13 @@ object CommonUtil {
fields: Seq[(String, String)],
varcharCols: Seq[String]
): Unit = {
- if (sortKey.diff(sortKey.distinct).length > 0 ||
- (sortKey.length > 1 && sortKey.contains(""))) {
+ val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+ if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+ (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
throw new MalformedCarbonCommandException(
"SORT_COLUMNS Either having duplicate columns : " +
- sortKey.diff(sortKey.distinct).mkString(",") + " or it contains illegal argumnet.")
+ sortKeyLowerCase.diff(sortKeyLowerCase.distinct).mkString(",") +
+ " or it contains illegal argument.")
Review comment:
`contains` to `contains an`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593857766
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2280/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593814993
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2275/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593242999
please add one test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386813122
##########
File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##########
@@ -760,11 +760,13 @@ object CommonUtil {
fields: Seq[(String, String)],
varcharCols: Seq[String]
): Unit = {
- if (sortKey.diff(sortKey.distinct).length > 0 ||
- (sortKey.length > 1 && sortKey.contains(""))) {
+ val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+ if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+ (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
throw new MalformedCarbonCommandException(
"SORT_COLUMNS Either having duplicate columns : " +
Review comment:
```suggestion
"SORT_COLUMNS Either contains duplicate columns : " +
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593304352
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2264/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386813002
##########
File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##########
@@ -760,11 +760,13 @@ object CommonUtil {
fields: Seq[(String, String)],
varcharCols: Seq[String]
): Unit = {
- if (sortKey.diff(sortKey.distinct).length > 0 ||
- (sortKey.length > 1 && sortKey.contains(""))) {
+ val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+ if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+ (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
throw new MalformedCarbonCommandException(
"SORT_COLUMNS Either having duplicate columns : " +
Review comment:
```suggestion
"SORT_COLUMNS Either having duplicate columns : " +
``` SORT_COLUMNS Either contains
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386813002
##########
File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##########
@@ -760,11 +760,13 @@ object CommonUtil {
fields: Seq[(String, String)],
varcharCols: Seq[String]
): Unit = {
- if (sortKey.diff(sortKey.distinct).length > 0 ||
- (sortKey.length > 1 && sortKey.contains(""))) {
+ val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+ if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+ (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
throw new MalformedCarbonCommandException(
"SORT_COLUMNS Either having duplicate columns : " +
Review comment:
```suggestion
"SORT_COLUMNS Either having duplicate columns : " +
``` SORT_COLUMNS Either contains
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] nihal0107 commented on issue #3647: [CARBONDATA-3348]
Fix the case insensitive validation for duplicate sort column.
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593856540
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593242779
add to whitelist
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] nihal0107 commented on a change in pull request #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on a change in pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386817816
##########
File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##########
@@ -760,11 +760,13 @@ object CommonUtil {
fields: Seq[(String, String)],
varcharCols: Seq[String]
): Unit = {
- if (sortKey.diff(sortKey.distinct).length > 0 ||
- (sortKey.length > 1 && sortKey.contains(""))) {
+ val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+ if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+ (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
throw new MalformedCarbonCommandException(
"SORT_COLUMNS Either having duplicate columns : " +
- sortKey.diff(sortKey.distinct).mkString(",") + " or it contains illegal argumnet.")
+ sortKeyLowerCase.diff(sortKeyLowerCase.distinct).mkString(",") +
+ " or it contains illegal argument.")
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593260243
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2258/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593211815
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] nihal0107 commented on a change in pull request #3647:
[CARBONDATA-3348] Fix the case insensitive validation for duplicate sort
column.
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on a change in pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386817767
##########
File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
##########
@@ -760,11 +760,13 @@ object CommonUtil {
fields: Seq[(String, String)],
varcharCols: Seq[String]
): Unit = {
- if (sortKey.diff(sortKey.distinct).length > 0 ||
- (sortKey.length > 1 && sortKey.contains(""))) {
+ val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+ if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+ (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
throw new MalformedCarbonCommandException(
"SORT_COLUMNS Either having duplicate columns : " +
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services