You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by GitBox <gi...@apache.org> on 2022/01/24 13:24:12 UTC

[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4248: Apply local sort task level property for insert

ShreelekhyaG opened a new pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248


    ### Why is this PR needed?
    Currently, When `carbon.partition.data.on.tasklevel` is enabled with local sort, the number of tasks launched for load will be based on node locality. But for insert command, the local sort task level property is not applied which is causing the number of tasks launched=number of partitions.
    
    ### What changes were proposed in this PR?
   Included changes to apply  `carbon.partition.data.on.tasklevel` property for insert command as well. Used `DataLoadCoalescedRDD` to coalesce the partitions and a `convertRDD `to unwrap partitions from `DataLoadPartitionWrap `and  iterate.
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - Yes
   
       
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020133644


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4459/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1029725152


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4472/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1039035831


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1039215456


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/611/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1039215456


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/611/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1038848529


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/610/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027821649


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4467/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1029738235


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/607/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ydvpankaj99 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ydvpankaj99 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020877629


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ydvpankaj99 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ydvpankaj99 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020877629






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1022950358


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6208/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027145136


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6210/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028850797


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6215/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1029636614


   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028021964


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6212/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1038830183


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4475/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#discussion_r790921451



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CommonLoadUtils.scala
##########
@@ -982,9 +985,29 @@ object CommonLoadUtils {
           persistedRDD = persistedRDDLocal
           transformedPlan
         } else {
+          val partitionBasedOnLocality = CarbonProperties.getInstance()

Review comment:
       can move this to above and reuse in the else case also




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1021193225


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4463/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1029722694


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6217/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028267760


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/604/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027169101


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/601/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020111186


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4458/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020108756


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6202/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028242502


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4469/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027150448


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4466/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028058656


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/603/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1038733004


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6219/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ShreelekhyaG commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028612368


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] kunal642 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
kunal642 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1029727517


   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1038741538


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] asfgit closed pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020893248


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6205/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020893248






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ydvpankaj99 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ydvpankaj99 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1021074881


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027844615


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/602/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020950127


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4461/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1021186442


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6207/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1021213617


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/598/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027811054


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6211/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028858952


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4471/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028024007


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4468/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1025861712


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6209/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1038864362


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6220/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1039182175


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4476/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028688303


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6214/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1022951088


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4464/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028704692


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/605/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028240080


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6213/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028874661


   Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/606/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020107875


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/593/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020130698


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6203/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020129726


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/594/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1021013153


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/596/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1029636667


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1025869883


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4465/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#discussion_r798221604



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CommonLoadUtils.scala
##########
@@ -1208,3 +1228,46 @@ object CommonLoadUtils {
     segmentMetaDataInfo
   }
 }
+
+/**
+ *  It unwraps partition from DataLoadPartitionWrap
+ */
+class DataLoadCoalescedUnwrapRDD[T: ClassTag](@transient var prev: RDD[T])

Review comment:
       please add comments for this wrapper class for better understanding




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on a change in pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#discussion_r798219869



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CommonLoadUtils.scala
##########
@@ -1208,3 +1228,33 @@ object CommonLoadUtils {
     segmentMetaDataInfo
   }
 }
+
+/**
+ *  It unwraps partition from DataLoadPartitionWrap
+ */
+class ConvertRDD[T: ClassTag](@transient var prev: RDD[T]) extends RDD[InternalRow](prev) {

Review comment:
       done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1039177020


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6221/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] Indhumathi27 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1038640886


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] asfgit closed pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ShreelekhyaG commented on pull request #4248: [WIP] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1020120003


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1028691469


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4470/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on a change in pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#discussion_r798470321



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CommonLoadUtils.scala
##########
@@ -1208,3 +1228,46 @@ object CommonLoadUtils {
     segmentMetaDataInfo
   }
 }
+
+/**
+ *  It unwraps partition from DataLoadPartitionWrap
+ */
+class DataLoadCoalescedUnwrapRDD[T: ClassTag](@transient var prev: RDD[T])

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] ShreelekhyaG commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
ShreelekhyaG commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1027899915


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1025923043


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/600/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#issuecomment-1022967108


   Build Failed  with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/599/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [carbondata] kunal642 commented on a change in pull request #4248: [CARBONDATA-4322] Apply local sort task level property for insert

Posted by GitBox <gi...@apache.org>.
kunal642 commented on a change in pull request #4248:
URL: https://github.com/apache/carbondata/pull/4248#discussion_r793277711



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CommonLoadUtils.scala
##########
@@ -1208,3 +1228,33 @@ object CommonLoadUtils {
     segmentMetaDataInfo
   }
 }
+
+/**
+ *  It unwraps partition from DataLoadPartitionWrap
+ */
+class ConvertRDD[T: ClassTag](@transient var prev: RDD[T]) extends RDD[InternalRow](prev) {

Review comment:
       give a more meaningful name to this RDD




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org