You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "Zouxxyy (via GitHub)" <gi...@apache.org> on 2025/01/09 02:03:24 UTC
[PR] [core] Fix remove orphan files with data file path directory [paimon]
Zouxxyy opened a new pull request, #4871:
URL: https://github.com/apache/paimon/pull/4871
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
Fix remove orphan files with data file path directory
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix remove orphan files with data file path directory [paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on code in PR #4871:
URL: https://github.com/apache/paimon/pull/4871#discussion_r1908131875
##########
paimon-core/src/main/java/org/apache/paimon/utils/FileStorePathFactory.java:
##########
@@ -37,6 +37,17 @@
@ThreadSafe
public class FileStorePathFactory {
+ public static final String MANIFEST_PATH = "manifest";
+ public static final String MANIFEST_PREFIX = "manifest-";
+ public static final String MANIFEST_LIST_PREFIX = "manifest-list-";
+ public static final String INDEX_MANIFEST_PREFIX = "index-manifest-";
+
+ public static final String INDEX_PATH = "index";
+ public static final String INDEX_PREFIX = "index-";
+
+ public static final String STATISTICS_PATH = "statistics";
+ public static final String STATISTICS_PREFIX = "stat-";
+
Review Comment:
I think both public and private are fine, and the BUCKET_PATH_PREFIX is also public.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix remove orphan files with data file path directory [paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on code in PR #4871:
URL: https://github.com/apache/paimon/pull/4871#discussion_r1908121315
##########
paimon-core/src/main/java/org/apache/paimon/operation/OrphanFilesClean.java:
##########
@@ -252,12 +253,14 @@ protected void collectWithoutDataFileWithManifestFlag(
/** List directories that contains data files and manifest files. */
protected List<Path> listPaimonFileDirs() {
+ FileStorePathFactory pathFactory = table.store().pathFactory();
Review Comment:
> Fix? Could you provide a detailed description of the bug?
See line 260
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix remove orphan files with data file path directory [paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on code in PR #4871:
URL: https://github.com/apache/paimon/pull/4871#discussion_r1908121315
##########
paimon-core/src/main/java/org/apache/paimon/operation/OrphanFilesClean.java:
##########
@@ -252,12 +253,14 @@ protected void collectWithoutDataFileWithManifestFlag(
/** List directories that contains data files and manifest files. */
protected List<Path> listPaimonFileDirs() {
+ FileStorePathFactory pathFactory = table.store().pathFactory();
Review Comment:
> Fix? Could you provide a detailed description of the bug?
See line 263, pathFactory.dataFilePath()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix remove orphan files with data file path directory [paimon]
Posted by "wwj6591812 (via GitHub)" <gi...@apache.org>.
wwj6591812 commented on PR #4871:
URL: https://github.com/apache/paimon/pull/4871#issuecomment-2579433989
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix remove orphan files with data file path directory [paimon]
Posted by "wwj6591812 (via GitHub)" <gi...@apache.org>.
wwj6591812 commented on code in PR #4871:
URL: https://github.com/apache/paimon/pull/4871#discussion_r1908108515
##########
paimon-core/src/main/java/org/apache/paimon/operation/OrphanFilesClean.java:
##########
@@ -252,12 +253,14 @@ protected void collectWithoutDataFileWithManifestFlag(
/** List directories that contains data files and manifest files. */
protected List<Path> listPaimonFileDirs() {
+ FileStorePathFactory pathFactory = table.store().pathFactory();
Review Comment:
Fix? Could you provide a detailed description of the bug?
##########
paimon-core/src/main/java/org/apache/paimon/utils/FileStorePathFactory.java:
##########
@@ -37,6 +37,17 @@
@ThreadSafe
public class FileStorePathFactory {
+ public static final String MANIFEST_PATH = "manifest";
+ public static final String MANIFEST_PREFIX = "manifest-";
+ public static final String MANIFEST_LIST_PREFIX = "manifest-list-";
+ public static final String INDEX_MANIFEST_PREFIX = "index-manifest-";
+
+ public static final String INDEX_PATH = "index";
+ public static final String INDEX_PREFIX = "index-";
+
+ public static final String STATISTICS_PATH = "statistics";
+ public static final String STATISTICS_PREFIX = "stat-";
+
Review Comment:
private
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix remove orphan files with data file path directory [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #4871:
URL: https://github.com/apache/paimon/pull/4871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org