You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by GitBox <gi...@apache.org> on 2021/09/21 12:48:10 UTC
[GitHub] [carbondata] nihal0107 opened a new pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
nihal0107 opened a new pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220
### Why is this PR needed?
Deleted records are not deleting and displaying in the select queries from the Alter added carbon segments after delete, update operations.
### What changes were proposed in this PR?
After delete/update operation on added segment, horizontal compaction will be triggered. In case of horizontal compation for external segment, delete delta file should be written at the place where segment present at place of default path.
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
### Is any new testcase added?
- Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-930350976
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5963/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924654619
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5943/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937660092
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6017/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r718535286
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,27 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ for (LoadMetadataDetails details: segmentDetails) {
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924046597
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/342/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937504525
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r718535286
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,27 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ for (LoadMetadataDetails details: segmentDetails) {
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937504525
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4270/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r719406411
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,28 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ if (segment.isExternalSegment()) {
+ for (LoadMetadataDetails details : segmentDetails) {
+ if (details.getLoadName().equals(segment.getSegmentNo())) {
+ segmentPath = details.getPath();
+ break;
+ }
+ }
+ } else {
+ segmentPath = CarbonTablePath.getSegmentPath(
+ identifier.getTablePath(), segment.getSegmentNo());
+ }
for (SegmentUpdateDetails block : updateDetails) {
if ((block.getBlockName().equalsIgnoreCase(blockName)) &&
(block.getSegmentName().equalsIgnoreCase(segment.getSegmentNo())) &&
!CarbonUpdateUtil.isBlockInvalid(block.getSegmentStatus())) {
Set<String> deltaFileTimestamps = block.getDeltaFileStamps();
if (deltaFileTimestamps != null && deltaFileTimestamps.size() > 0) {
+ String finalSegmentPath = segmentPath;
Review comment:
We are updating the same variable inside for loop so it is giving error as `Variable used in lambda expression should be final or effectively final`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924704989
retest this please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] kunal642 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
kunal642 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r718528557
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,27 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ for (LoadMetadataDetails details: segmentDetails) {
Review comment:
check whether the segment is external before this loop like this:
if (segment.isExternalSegment) {
for (LoadMetadataDetails details: segmentDetails) {
...
}
} else {
segmentPath = CarbonTablePath.getSegmentPath(
identifier.getTablePath(), segment.getSegmentNo());
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924809807
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5944/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-931292953
@nihal0107 in Why this PR section of PR description, it's just written that the delete is not working. But that's not enough, can you please mention the root cause also there? Then, in the next section, you add details of fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937442194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937507134
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6016/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-930328765
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/363/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-930329242
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4218/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r719406411
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,28 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ if (segment.isExternalSegment()) {
+ for (LoadMetadataDetails details : segmentDetails) {
+ if (details.getLoadName().equals(segment.getSegmentNo())) {
+ segmentPath = details.getPath();
+ break;
+ }
+ }
+ } else {
+ segmentPath = CarbonTablePath.getSegmentPath(
+ identifier.getTablePath(), segment.getSegmentNo());
+ }
for (SegmentUpdateDetails block : updateDetails) {
if ((block.getBlockName().equalsIgnoreCase(blockName)) &&
(block.getSegmentName().equalsIgnoreCase(segment.getSegmentNo())) &&
!CarbonUpdateUtil.isBlockInvalid(block.getSegmentStatus())) {
Set<String> deltaFileTimestamps = block.getDeltaFileStamps();
if (deltaFileTimestamps != null && deltaFileTimestamps.size() > 0) {
+ String finalSegmentPath = segmentPath;
Review comment:
We are updating the same variable inside for loop so scala is giving error as `Variable used in lambda expression should be final or effectively final`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924579308
retest this please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924645854
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924644181
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4197/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-931302728
> @nihal0107 in Why this PR section of PR description, it's just written that the delete is not working. But that's not enough, can you please mention the root cause also there? Then, in the next section, you add details of fix.
@akashrn5 added the details description and root cause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-935499072
@akashrn5 @kunal642 can we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] kunal642 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
kunal642 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r718528557
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,27 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ for (LoadMetadataDetails details: segmentDetails) {
Review comment:
check whether the segment is external before this loop like this:
if (segment.isExternalSegment) {
for (LoadMetadataDetails details: segmentDetails) {
...
}
} else {
segmentPath = CarbonTablePath.getSegmentPath(
identifier.getTablePath(), segment.getSegmentNo());
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937652922
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/417/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924809325
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/345/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924816633
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4198/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-930328765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924051339
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5941/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937506978
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/416/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-924057888
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4195/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937442194
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#discussion_r719397599
##########
File path: core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
##########
@@ -366,17 +366,28 @@ public boolean accept(CarbonFile pathName) {
*/
public List<String> getDeleteDeltaFilesList(final Segment segment, final String blockName) {
List<String> deleteDeltaFileList = new ArrayList<>();
- String segmentPath = CarbonTablePath.getSegmentPath(
- identifier.getTablePath(), segment.getSegmentNo());
+ String segmentPath = null;
+ if (segment.isExternalSegment()) {
+ for (LoadMetadataDetails details : segmentDetails) {
+ if (details.getLoadName().equals(segment.getSegmentNo())) {
+ segmentPath = details.getPath();
+ break;
+ }
+ }
+ } else {
+ segmentPath = CarbonTablePath.getSegmentPath(
+ identifier.getTablePath(), segment.getSegmentNo());
+ }
for (SegmentUpdateDetails block : updateDetails) {
if ((block.getBlockName().equalsIgnoreCase(blockName)) &&
(block.getSegmentName().equalsIgnoreCase(segment.getSegmentNo())) &&
!CarbonUpdateUtil.isBlockInvalid(block.getSegmentStatus())) {
Set<String> deltaFileTimestamps = block.getDeltaFileStamps();
if (deltaFileTimestamps != null && deltaFileTimestamps.size() > 0) {
+ String finalSegmentPath = segmentPath;
Review comment:
why one more variable is required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937668408
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4273/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
kunal642 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937691672
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] nihal0107 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
nihal0107 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937548745
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] kunal642 commented on pull request #4220: [CARBONDATA-4228] [CARBONDATA-4203] Fixed update/delete after alter add segment
Posted by GitBox <gi...@apache.org>.
kunal642 commented on pull request #4220:
URL: https://github.com/apache/carbondata/pull/4220#issuecomment-937691672
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org