You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2020/03/26 18:32:54 UTC
[GitHub] [carbondata] akashrn5 opened a new pull request #3683: [WIP]Fix
clean up issue with respect to segmentMetadaInfo
akashrn5 opened a new pull request #3683: [WIP]Fix clean up issue with respect to segmentMetadaInfo
URL: https://github.com/apache/carbondata/pull/3683
### Why is this PR needed?
### What changes were proposed in this PR?
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
### Is any new testcase added?
- No
- Yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3683: [WIP]Fix clean
up issue with respect to segmentMetadaInfo
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3683: [WIP]Fix clean up issue with respect to segmentMetadaInfo
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-604673845
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2569/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3683: [WIP]Fix clean
up issue with respect to segmentMetadaInfo
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3683: [WIP]Fix clean up issue with respect to segmentMetadaInfo
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-604680837
Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/861/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#discussion_r399380210
##########
File path: core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
##########
@@ -579,10 +580,28 @@ public static void cleanUpDeltaFiles(CarbonTable table, boolean forceDelete) thr
}
String UUID = String.valueOf(System.currentTimeMillis());
List<Segment> segmentFilesToBeUpdatedLatest = new ArrayList<>();
+ CarbonFile segmentFilesLocation =
+ FileFactory.getCarbonFile(CarbonTablePath.getSegmentFilesLocation(table.getTablePath()));
for (Segment segment : segmentFilesToBeUpdated) {
- String file =
- SegmentFileStore.writeSegmentFile(table, segment.getSegmentNo(), UUID);
- segmentFilesToBeUpdatedLatest.add(new Segment(segment.getSegmentNo(), file));
+ SegmentFileStore fileStore =
+ new SegmentFileStore(table.getTablePath(), segment.getSegmentFileName());
+ segment.setSegmentMetaDataInfo(fileStore.getSegmentFile().getSegmentMetaDataInfo());
Review comment:
added
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-605197577
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2579/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-605131953
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2575/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-605205781
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/871/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] kunal642 commented on issue #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
kunal642 commented on issue #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-608231160
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] ydvpankaj99 commented on issue #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
ydvpankaj99 commented on issue #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-605046083
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] asfgit closed pull request #3683: [CARBONDATA-3755]Fix
clean up issue with respect to segmentMetadaInfo after update and clean files
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] CarbonDataQA1 commented on issue #3683:
[CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after
update and clean files
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on issue #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#issuecomment-605130361
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/867/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request
#3683: [CARBONDATA-3755]Fix clean up issue with respect to
segmentMetadaInfo after update and clean files
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3683: [CARBONDATA-3755]Fix clean up issue with respect to segmentMetadaInfo after update and clean files
URL: https://github.com/apache/carbondata/pull/3683#discussion_r399261249
##########
File path: core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
##########
@@ -579,10 +580,28 @@ public static void cleanUpDeltaFiles(CarbonTable table, boolean forceDelete) thr
}
String UUID = String.valueOf(System.currentTimeMillis());
List<Segment> segmentFilesToBeUpdatedLatest = new ArrayList<>();
+ CarbonFile segmentFilesLocation =
+ FileFactory.getCarbonFile(CarbonTablePath.getSegmentFilesLocation(table.getTablePath()));
for (Segment segment : segmentFilesToBeUpdated) {
- String file =
- SegmentFileStore.writeSegmentFile(table, segment.getSegmentNo(), UUID);
- segmentFilesToBeUpdatedLatest.add(new Segment(segment.getSegmentNo(), file));
+ SegmentFileStore fileStore =
+ new SegmentFileStore(table.getTablePath(), segment.getSegmentFileName());
+ segment.setSegmentMetaDataInfo(fileStore.getSegmentFile().getSegmentMetaDataInfo());
Review comment:
Can you please add a testcase for this issue, if possible
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services