You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/02/15 19:47:21 UTC
[GitHub] [iceberg] danielcweeks opened a new pull request #2237: Use aws sdk v2 single object delete (not bulk delete) for S3FileIO
danielcweeks opened a new pull request #2237:
URL: https://github.com/apache/iceberg/pull/2237
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #2237: Use aws sdk v2 single object delete (not bulk delete) for S3FileIO
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #2237:
URL: https://github.com/apache/iceberg/pull/2237
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 commented on pull request #2237: Use aws sdk v2 single object delete (not bulk delete) for S3FileIO
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on pull request #2237:
URL: https://github.com/apache/iceberg/pull/2237#issuecomment-779675432
btw, there is a spelling mistake at https://github.com/apache/iceberg/blob/master/aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputFile.java#L60, could you also fix that when merging this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jacques-n commented on a change in pull request #2237: Use aws sdk v2 single object delete (not bulk delete) for S3FileIO
Posted by GitBox <gi...@apache.org>.
jacques-n commented on a change in pull request #2237:
URL: https://github.com/apache/iceberg/pull/2237#discussion_r576476162
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -88,12 +86,10 @@ public OutputFile newOutputFile(String path) {
@Override
public void deleteFile(String path) {
Review comment:
Is there a guarantee that the path isn't a "directory"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #2237: Use aws sdk v2 single object delete (not bulk delete) for S3FileIO
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2237:
URL: https://github.com/apache/iceberg/pull/2237#issuecomment-785263292
Thanks @danielcweeks!
@jackye1995, I didn't wait to fix that typo because I didn't want this commit to touch an additional unrelated file. We can fix that in a separate PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] danielcweeks commented on a change in pull request #2237: Use aws sdk v2 single object delete (not bulk delete) for S3FileIO
Posted by GitBox <gi...@apache.org>.
danielcweeks commented on a change in pull request #2237:
URL: https://github.com/apache/iceberg/pull/2237#discussion_r576483658
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -88,12 +86,10 @@ public OutputFile newOutputFile(String path) {
@Override
public void deleteFile(String path) {
Review comment:
There's really no concept of a directory (in the sense that you can recursively delete a bunch of data) in S3. Directories are just a byproduct of whatever you use as a delimiter in listing (most typically `/` to make it look like a filesystem structure). Some FileSystem implementation use a marker for empty directories (e.g. `<path>_$folder$`), but that's just to workaround the fact that there isn't such a thing.
For Iceberg, this should be safe as listing isn't used and files are all directly referenced.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org