You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "saurabhd336 (via GitHub)" <gi...@apache.org> on 2023/07/19 06:09:39 UTC
[GitHub] [pinot] saurabhd336 opened a new pull request, #11132: Sanitise API inputs used as file path variables
saurabhd336 opened a new pull request, #11132:
URL: https://github.com/apache/pinot/pull/11132
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #11132: Sanitise API inputs used as file path variables
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #11132:
URL: https://github.com/apache/pinot/pull/11132#discussion_r1270911318
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/LLCSegmentCompletionHandlers.java:
##########
@@ -448,8 +448,10 @@ private static File extractSegmentFromFormToLocalTempFile(FormDataMultiPart form
"Invalid multi-part for segment: %s", segmentName);
FormDataBodyPart bodyPart = map.values().iterator().next().get(0);
- File localTempFile = new File(ControllerFilePathProvider.getInstance().getFileUploadTempDir(),
- getTempSegmentFileName(segmentName));
+ File localTempFile = org.apache.pinot.common.utils.FileUtils.concatAndValidateFile(
Review Comment:
unfortunately this class already imported common-io FileUtils. it has to use fully qualified package name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] soumitra-st commented on a diff in pull request #11132: Sanitise API inputs used as file path variables
Posted by "soumitra-st (via GitHub)" <gi...@apache.org>.
soumitra-st commented on code in PR #11132:
URL: https://github.com/apache/pinot/pull/11132#discussion_r1270682560
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/LLCSegmentCompletionHandlers.java:
##########
@@ -448,8 +448,10 @@ private static File extractSegmentFromFormToLocalTempFile(FormDataMultiPart form
"Invalid multi-part for segment: %s", segmentName);
FormDataBodyPart bodyPart = map.values().iterator().next().get(0);
- File localTempFile = new File(ControllerFilePathProvider.getInstance().getFileUploadTempDir(),
- getTempSegmentFileName(segmentName));
+ File localTempFile = org.apache.pinot.common.utils.FileUtils.concatAndValidateFile(
Review Comment:
Shall we import org.apache.pinot.common.utils.FileUtils?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #11132: Sanitise API inputs used as file path variables
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #11132:
URL: https://github.com/apache/pinot/pull/11132#issuecomment-1641509677
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#11132](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (9b74ea4) into [master](https://app.codecov.io/gh/apache/pinot/commit/8fff7eb70597eb74c05ee788378eb8fa1ed8efaf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (8fff7eb) will **increase** coverage by `0.00%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #11132 +/- ##
=========================================
Coverage 0.11% 0.11%
=========================================
Files 2203 2148 -55
Lines 118214 115715 -2499
Branches 17891 17593 -298
=========================================
Hits 137 137
+ Misses 118057 115558 -2499
Partials 20 20
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1temurin11 | `?` | |
| integration1temurin17 | `?` | |
| integration1temurin20 | `?` | |
| integration2temurin11 | `?` | |
| integration2temurin20 | `?` | |
| unittests1temurin17 | `?` | |
| unittests1temurin20 | `?` | |
| unittests2temurin11 | `?` | |
| unittests2temurin17 | `0.11% <0.00%> (-0.01%)` | :arrow_down: |
| unittests2temurin20 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...e/pinot/common/function/TransformFunctionType.java](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...er/api/resources/LLCSegmentCompletionHandlers.java](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL0xMQ1NlZ21lbnRDb21wbGV0aW9uSGFuZGxlcnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...ces/PinotSegmentUploadDownloadRestletResource.java](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90U2VnbWVudFVwbG9hZERvd25sb2FkUmVzdGxldFJlc291cmNlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...che/pinot/controller/util/FileIngestionHelper.java](https://app.codecov.io/gh/apache/pinot/pull/11132?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci91dGlsL0ZpbGVJbmdlc3Rpb25IZWxwZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
... and [57 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/11132/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on a diff in pull request #11132: Sanitise API inputs used as file path variables
Posted by "walterddr (via GitHub)" <gi...@apache.org>.
walterddr commented on code in PR #11132:
URL: https://github.com/apache/pinot/pull/11132#discussion_r1270001646
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/LLCSegmentCompletionHandlers.java:
##########
@@ -448,8 +448,13 @@ private static File extractSegmentFromFormToLocalTempFile(FormDataMultiPart form
"Invalid multi-part for segment: %s", segmentName);
FormDataBodyPart bodyPart = map.values().iterator().next().get(0);
- File localTempFile = new File(ControllerFilePathProvider.getInstance().getFileUploadTempDir(),
- getTempSegmentFileName(segmentName));
+ File fileUploadTempDir = ControllerFilePathProvider.getInstance().getFileUploadTempDir();
+ File localTempFile = new File(fileUploadTempDir, getTempSegmentFileName(segmentName));
+
+ if (!localTempFile.getCanonicalPath().startsWith(fileUploadTempDir.getPath())) {
+ throw new IllegalArgumentException("Invalid segment name: " + segmentName);
+ }
Review Comment:
create a file path util (put in pinot-common/src/main/java/org/apache/pinot/common/utils/FileUtils.java)
```suggestion
public static File concatAndSanitizedFile(File folderDir, File filename) {
File filePath = new File(folderDir, filename);
if (!filePath.getCanonicalPath().startsWith(folderDir.getPath())) {
throw new IllegalArgumentException("Invalid file name: " + filename);
}
}
```
and then the util can be used across all the following
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on pull request #11132: Sanitise API inputs used as file path variables
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on PR #11132:
URL: https://github.com/apache/pinot/pull/11132#issuecomment-1641474140
cc: @soumitra-st
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 merged pull request #11132: Sanitise API inputs used as file path variables
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 merged PR #11132:
URL: https://github.com/apache/pinot/pull/11132
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #11132: Sanitise API inputs used as file path variables
Posted by "saurabhd336 (via GitHub)" <gi...@apache.org>.
saurabhd336 commented on code in PR #11132:
URL: https://github.com/apache/pinot/pull/11132#discussion_r1270323609
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/LLCSegmentCompletionHandlers.java:
##########
@@ -448,8 +448,13 @@ private static File extractSegmentFromFormToLocalTempFile(FormDataMultiPart form
"Invalid multi-part for segment: %s", segmentName);
FormDataBodyPart bodyPart = map.values().iterator().next().get(0);
- File localTempFile = new File(ControllerFilePathProvider.getInstance().getFileUploadTempDir(),
- getTempSegmentFileName(segmentName));
+ File fileUploadTempDir = ControllerFilePathProvider.getInstance().getFileUploadTempDir();
+ File localTempFile = new File(fileUploadTempDir, getTempSegmentFileName(segmentName));
+
+ if (!localTempFile.getCanonicalPath().startsWith(fileUploadTempDir.getPath())) {
+ throw new IllegalArgumentException("Invalid segment name: " + segmentName);
+ }
Review Comment:
Makes sense. Addressed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org