You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "tzuan16 (via GitHub)" <gi...@apache.org> on 2024/10/18 19:48:33 UTC
[PR] Fix Azure path comparison when filePath is root ("/") [polaris]
tzuan16 opened a new pull request, #385:
URL: https://github.com/apache/polaris/pull/385
# Description
The `AzureLocation.isChildOf` method is currently broken if the parent path points to the root of the container (e.g. `abfss://container@acc.blob.core.windows.net` or `abfss://container@acc.blob.core.windows.net/`). The `filePath` of the path in this case would be parsed as `/` since we always add a "trailing slash". Now if the comparing child path looks something like `abfss://container@acc.blob.core.windows.net/some/file/path/metadata`, the `filePath` would be parsed as `some/file/path/metadata/` after applying the trailing slash, leading to the failure of the `childFilePath.startsWith(parentFilePath)` check since the childFilePath doesn't start with a slash.
Fix is to always ensure a leading slash on top of ensuring a trailing slash.
## Type of change
Please delete options that are not relevant.
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Documentation update
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update
# How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
- [ ] AzureLocationTest unit test
- [ ] Test B
**Test Configuration**:
* Hardware:
* Toolchain:
* SDK:
# Checklist:
Please delete options that are not relevant.
- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] If adding new functionality, I have discussed my implementation with the community using the linked GitHub issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #385:
URL: https://github.com/apache/polaris/pull/385#discussion_r1806978333
##########
polaris-core/src/main/java/org/apache/polaris/core/storage/azure/AzureLocation.java:
##########
@@ -108,15 +108,24 @@ public boolean isChildOf(@NotNull StorageLocation potentialParent) {
AzureLocation potentialAzureParent = (AzureLocation) potentialParent;
if (this.container.equals(potentialAzureParent.container)) {
if (this.storageAccount.equals(potentialAzureParent.storageAccount)) {
- String slashTerminatedFilePath = ensureTrailingSlash(this.filePath);
- String slashTerminatedParentFilePath = ensureTrailingSlash(potentialAzureParent.filePath);
- return slashTerminatedFilePath.startsWith(slashTerminatedParentFilePath);
+ String formattedFilePath = ensureLeadingSlash(ensureTrailingSlash(this.filePath));
+ String formattedParentFilePath = ensureLeadingSlash(ensureTrailingSlash(potentialAzureParent.filePath));
+ return formattedFilePath.startsWith(formattedParentFilePath);
}
}
}
return false;
}
+ /** If a path doesn't start with `/`, this will add one */
+ private @NotNull String ensureLeadingSlash(@NotNull String location) {
Review Comment:
Ah, I see... that's right. All this manual-path-parsing stuff is so annoying. Can we just move this up to `StorageLocation` then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard merged PR #385:
URL: https://github.com/apache/polaris/pull/385
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #385:
URL: https://github.com/apache/polaris/pull/385#discussion_r1806961285
##########
polaris-core/src/main/java/org/apache/polaris/core/storage/azure/AzureLocation.java:
##########
@@ -108,15 +108,24 @@ public boolean isChildOf(@NotNull StorageLocation potentialParent) {
AzureLocation potentialAzureParent = (AzureLocation) potentialParent;
if (this.container.equals(potentialAzureParent.container)) {
if (this.storageAccount.equals(potentialAzureParent.storageAccount)) {
- String slashTerminatedFilePath = ensureTrailingSlash(this.filePath);
- String slashTerminatedParentFilePath = ensureTrailingSlash(potentialAzureParent.filePath);
- return slashTerminatedFilePath.startsWith(slashTerminatedParentFilePath);
+ String formattedFilePath = ensureLeadingSlash(ensureTrailingSlash(this.filePath));
+ String formattedParentFilePath = ensureLeadingSlash(ensureTrailingSlash(potentialAzureParent.filePath));
+ return formattedFilePath.startsWith(formattedParentFilePath);
}
}
}
return false;
}
+ /** If a path doesn't start with `/`, this will add one */
+ private @NotNull String ensureLeadingSlash(@NotNull String location) {
Review Comment:
Since we always call this along with `ensureTrailingSlash` can we just combine them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "tzuan16 (via GitHub)" <gi...@apache.org>.
tzuan16 commented on code in PR #385:
URL: https://github.com/apache/polaris/pull/385#discussion_r1806967090
##########
polaris-core/src/main/java/org/apache/polaris/core/storage/azure/AzureLocation.java:
##########
@@ -108,15 +108,24 @@ public boolean isChildOf(@NotNull StorageLocation potentialParent) {
AzureLocation potentialAzureParent = (AzureLocation) potentialParent;
if (this.container.equals(potentialAzureParent.container)) {
if (this.storageAccount.equals(potentialAzureParent.storageAccount)) {
- String slashTerminatedFilePath = ensureTrailingSlash(this.filePath);
- String slashTerminatedParentFilePath = ensureTrailingSlash(potentialAzureParent.filePath);
- return slashTerminatedFilePath.startsWith(slashTerminatedParentFilePath);
+ String formattedFilePath = ensureLeadingSlash(ensureTrailingSlash(this.filePath));
+ String formattedParentFilePath = ensureLeadingSlash(ensureTrailingSlash(potentialAzureParent.filePath));
+ return formattedFilePath.startsWith(formattedParentFilePath);
}
}
}
return false;
}
+ /** If a path doesn't start with `/`, this will add one */
+ private @NotNull String ensureLeadingSlash(@NotNull String location) {
Review Comment:
`ensureTrailingSlash` is also called from the parent class `StorageLocation`, so I wonder if we should just keep them separate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "tzuan16 (via GitHub)" <gi...@apache.org>.
tzuan16 commented on code in PR #385:
URL: https://github.com/apache/polaris/pull/385#discussion_r1806970513
##########
polaris-core/src/main/java/org/apache/polaris/core/storage/azure/AzureLocation.java:
##########
@@ -108,15 +108,24 @@ public boolean isChildOf(@NotNull StorageLocation potentialParent) {
AzureLocation potentialAzureParent = (AzureLocation) potentialParent;
if (this.container.equals(potentialAzureParent.container)) {
if (this.storageAccount.equals(potentialAzureParent.storageAccount)) {
- String slashTerminatedFilePath = ensureTrailingSlash(this.filePath);
- String slashTerminatedParentFilePath = ensureTrailingSlash(potentialAzureParent.filePath);
- return slashTerminatedFilePath.startsWith(slashTerminatedParentFilePath);
+ String formattedFilePath = ensureLeadingSlash(ensureTrailingSlash(this.filePath));
+ String formattedParentFilePath = ensureLeadingSlash(ensureTrailingSlash(potentialAzureParent.filePath));
+ return formattedFilePath.startsWith(formattedParentFilePath);
}
}
}
return false;
}
+ /** If a path doesn't start with `/`, this will add one */
+ private @NotNull String ensureLeadingSlash(@NotNull String location) {
Review Comment:
No, the `StorageLocation` class does the isChildOf comparison by comparing the _full_ location path, so it doesn't make sense to change something like `s3://....` to `/s3://....`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "eric-maynard (via GitHub)" <gi...@apache.org>.
eric-maynard commented on code in PR #385:
URL: https://github.com/apache/polaris/pull/385#discussion_r1806968857
##########
polaris-core/src/main/java/org/apache/polaris/core/storage/azure/AzureLocation.java:
##########
@@ -108,15 +108,24 @@ public boolean isChildOf(@NotNull StorageLocation potentialParent) {
AzureLocation potentialAzureParent = (AzureLocation) potentialParent;
if (this.container.equals(potentialAzureParent.container)) {
if (this.storageAccount.equals(potentialAzureParent.storageAccount)) {
- String slashTerminatedFilePath = ensureTrailingSlash(this.filePath);
- String slashTerminatedParentFilePath = ensureTrailingSlash(potentialAzureParent.filePath);
- return slashTerminatedFilePath.startsWith(slashTerminatedParentFilePath);
+ String formattedFilePath = ensureLeadingSlash(ensureTrailingSlash(this.filePath));
+ String formattedParentFilePath = ensureLeadingSlash(ensureTrailingSlash(potentialAzureParent.filePath));
+ return formattedFilePath.startsWith(formattedParentFilePath);
}
}
}
return false;
}
+ /** If a path doesn't start with `/`, this will add one */
+ private @NotNull String ensureLeadingSlash(@NotNull String location) {
Review Comment:
Do we not need to `ensureLeadingSlash` in `StorageLocation`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Azure path comparison when filePath is root ("/") [polaris]
Posted by "tzuan16 (via GitHub)" <gi...@apache.org>.
tzuan16 commented on code in PR #385:
URL: https://github.com/apache/polaris/pull/385#discussion_r1807010480
##########
polaris-core/src/main/java/org/apache/polaris/core/storage/azure/AzureLocation.java:
##########
@@ -108,15 +108,24 @@ public boolean isChildOf(@NotNull StorageLocation potentialParent) {
AzureLocation potentialAzureParent = (AzureLocation) potentialParent;
if (this.container.equals(potentialAzureParent.container)) {
if (this.storageAccount.equals(potentialAzureParent.storageAccount)) {
- String slashTerminatedFilePath = ensureTrailingSlash(this.filePath);
- String slashTerminatedParentFilePath = ensureTrailingSlash(potentialAzureParent.filePath);
- return slashTerminatedFilePath.startsWith(slashTerminatedParentFilePath);
+ String formattedFilePath = ensureLeadingSlash(ensureTrailingSlash(this.filePath));
+ String formattedParentFilePath = ensureLeadingSlash(ensureTrailingSlash(potentialAzureParent.filePath));
+ return formattedFilePath.startsWith(formattedParentFilePath);
}
}
}
return false;
}
+ /** If a path doesn't start with `/`, this will add one */
+ private @NotNull String ensureLeadingSlash(@NotNull String location) {
Review Comment:
Sounds good. Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org