You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "czy006 (via GitHub)" <gi...@apache.org> on 2025/12/25 07:31:52 UTC
[PR] Draft: [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
czy006 opened a new pull request, #4024:
URL: https://github.com/apache/amoro/pull/4024
Draft: [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module (#4022)
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://amoro.apache.org/how-to-contribute/
2. If the PR is related to an issue in https://github.com/apache/amoro/issues, add '[AMORO-XXXX]' in your PR title, e.g., '[AMORO-XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][AMORO-XXXX] Your PR title ...'.
-->
## Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about its use case.
2. If you fix a bug, you can clarify why it is a bug.
3. Use Fix/Resolve/Close #{ISSUE_NUMBER} to link this PR to its related issue
-->
Close #4022 .
## Brief change log
<!--
Clearly describe the changes made in modules, classes, methods, etc.
-->
-
## How was this patch tested?
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "czy006 (via GitHub)" <gi...@apache.org>.
czy006 commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2660861130
##########
amoro-common/src/main/java/org/apache/amoro/maintainer/MaintainerMetrics.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.maintainer;
+
+/**
+ * Metrics collector interface for table maintenance operations. Implementations can collect metrics
+ * to different monitoring systems.
+ */
+public interface MaintainerMetrics {
Review Comment:
this name same to **org/apache/amoro/metrics/promethues/MetricsCollector.java**,These two are not the same. In the future, these indicators will be reportable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Draft: [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #4024:
URL: https://github.com/apache/amoro/pull/4024#issuecomment-3691116813
## [Codecov](https://app.codecov.io/gh/apache/amoro/pull/4024?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `247 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 22.44%. Comparing base ([`cbdc517`](https://app.codecov.io/gh/apache/amoro/commit/cbdc5176b760b2ab46daa5051f29b72f1fee7175?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`19c989f`](https://app.codecov.io/gh/apache/amoro/commit/19c989f1392c614cc29c96fd591407553cca316d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 20 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/amoro/pull/4024?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../amoro/formats/iceberg/utils/IcebergTableUtil.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-format-iceberg%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fformats%2Ficeberg%2Futils%2FIcebergTableUtil.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tZm9ybWF0LWljZWJlcmcvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Ftb3JvL2Zvcm1hdHMvaWNlYmVyZy91dGlscy9JY2ViZXJnVGFibGVVdGlsLmphdmE=) | 0.00% | [80 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...moro/formats/iceberg/utils/RollingFileCleaner.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-format-iceberg%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fformats%2Ficeberg%2Futils%2FRollingFileCleaner.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tZm9ybWF0LWljZWJlcmcvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Ftb3JvL2Zvcm1hdHMvaWNlYmVyZy91dGlscy9Sb2xsaW5nRmlsZUNsZWFuZXIuamF2YQ==) | 0.00% | [60 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ats/iceberg/maintainer/IcebergTableMaintainer.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-format-iceberg%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fformats%2Ficeberg%2Fmaintainer%2FIcebergTableMaintainer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tZm9ybWF0LWljZWJlcmcvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Ftb3JvL2Zvcm1hdHMvaWNlYmVyZy9tYWludGFpbmVyL0ljZWJlcmdUYWJsZU1haW50YWluZXIuamF2YQ==) | 0.00% | [54 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rmats/iceberg/maintainer/MixedTableMaintainer.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-format-iceberg%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fformats%2Ficeberg%2Fmaintainer%2FMixedTableMaintainer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tZm9ybWF0LWljZWJlcmcvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Ftb3JvL2Zvcm1hdHMvaWNlYmVyZy9tYWludGFpbmVyL01peGVkVGFibGVNYWludGFpbmVyLmphdmE=) | 0.00% | [44 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...org/apache/amoro/maintainer/MaintainerMetrics.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fmaintainer%2FMaintainerMetrics.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9hbW9yby9tYWludGFpbmVyL01haW50YWluZXJNZXRyaWNzLmphdmE=) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...va/org/apache/amoro/maintainer/OptimizingInfo.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fmaintainer%2FOptimizingInfo.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9hbW9yby9tYWludGFpbmVyL09wdGltaXppbmdJbmZvLmphdmE=) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pache/amoro/maintainer/TableMaintainerContext.java](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&filepath=amoro-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Famoro%2Fmaintainer%2FTableMaintainerContext.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YW1vcm8tY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9hbW9yby9tYWludGFpbmVyL1RhYmxlTWFpbnRhaW5lckNvbnRleHQuamF2YQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/amoro/pull/4024?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #4024 +/- ##
============================================
+ Coverage 22.12% 22.44% +0.31%
- Complexity 2461 2554 +93
============================================
Files 445 458 +13
Lines 40897 42065 +1168
Branches 5767 5909 +142
============================================
+ Hits 9050 9441 +391
- Misses 31089 31812 +723
- Partials 758 812 +54
```
| [Flag](https://app.codecov.io/gh/apache/amoro/pull/4024/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [trino](https://app.codecov.io/gh/apache/amoro/pull/4024/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `22.44% <0.00%> (+0.31%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/amoro/pull/4024?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "czy006 (via GitHub)" <gi...@apache.org>.
czy006 commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2660870991
##########
amoro-ams/src/main/java/org/apache/amoro/server/optimizing/maintainer/TableMaintainers.java:
##########
@@ -21,25 +21,39 @@
import org.apache.amoro.AmoroTable;
import org.apache.amoro.TableFormat;
import org.apache.amoro.TableRuntime;
+import org.apache.amoro.formats.iceberg.maintainer.MixedTableMaintainer;
+import org.apache.amoro.maintainer.TableMaintainer;
import org.apache.amoro.server.table.DefaultTableRuntime;
-import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
import org.apache.amoro.table.MixedTable;
import org.apache.iceberg.Table;
/** Factory for creating {@link TableMaintainer}. */
public class TableMaintainers {
- /** Create a {@link TableMaintainer} for the given table. */
+ /**
+ * Create a {@link TableMaintainer} for the given table.
+ *
+ * @deprecated Use {@link TableMaintainerFactory#create(AmoroTable, TableRuntime)} instead.
Review Comment:
It will be removed in the next version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "baiyangtx (via GitHub)" <gi...@apache.org>.
baiyangtx commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2652871555
##########
amoro-ams/src/main/java/org/apache/amoro/server/optimizing/maintainer/TableMaintainerFactory.java:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.server.optimizing.maintainer;
+
+import org.apache.amoro.AmoroTable;
+import org.apache.amoro.TableFormat;
+import org.apache.amoro.formats.iceberg.maintainer.IcebergTableMaintainer;
+import org.apache.amoro.formats.iceberg.maintainer.MixedTableMaintainer;
+import org.apache.amoro.maintainer.TableMaintainer;
+import org.apache.amoro.server.table.DefaultTableRuntime;
+import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
+import org.apache.amoro.table.MixedTable;
+import org.apache.iceberg.Table;
+
+/** Factory for creating {@link TableMaintainer} instances. */
+public class TableMaintainerFactory {
+
+ /**
+ * Create an Iceberg table maintainer with AMS context.
+ *
+ * @param table the Iceberg table
+ * @param tableRuntime the AMS table runtime
+ * @return IcebergTableMaintainer instance
+ */
+ public static IcebergTableMaintainer createIcebergMaintainer(
+ Table table, DefaultTableRuntime tableRuntime) {
+ return new IcebergTableMaintainer(
+ table,
+ tableRuntime.getTableIdentifier().getIdentifier(),
+ new DefaultTableMaintainerContext(tableRuntime));
+ }
+
+ /**
+ * Create a {@link TableMaintainer} for the given table.
+ *
+ * @param amoroTable the Amoro table
+ * @param tableRuntime the table runtime
+ * @return TableMaintainer instance
+ */
+ public static TableMaintainer create(AmoroTable<?> amoroTable, Object tableRuntime) {
Review Comment:
```suggestion
public static TableMaintainer create(AmoroTable<?> amoroTable, TableRuntime tableRuntime) {
```
##########
amoro-ams/src/main/java/org/apache/amoro/server/optimizing/maintainer/TableMaintainers.java:
##########
@@ -21,25 +21,42 @@
import org.apache.amoro.AmoroTable;
import org.apache.amoro.TableFormat;
import org.apache.amoro.TableRuntime;
+import org.apache.amoro.formats.iceberg.maintainer.MixedTableMaintainer;
+import org.apache.amoro.maintainer.TableMaintainer;
import org.apache.amoro.server.table.DefaultTableRuntime;
-import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
import org.apache.amoro.table.MixedTable;
import org.apache.iceberg.Table;
/** Factory for creating {@link TableMaintainer}. */
+@Deprecated
Review Comment:
Maybe we can remove this class directly?
##########
amoro-ams/src/test/java/org/apache/amoro/server/optimizing/maintainer/TestDataExpire.java:
##########
@@ -31,6 +31,8 @@
import org.apache.amoro.data.ChangeAction;
import org.apache.amoro.data.DataFileType;
import org.apache.amoro.data.PrimaryKeyedFile;
+import org.apache.amoro.formats.iceberg.maintainer.IcebergTableMaintainer;
Review Comment:
Can we move those Tests to iceberg-format module?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "czy006 (via GitHub)" <gi...@apache.org>.
czy006 commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2661572601
##########
amoro-common/src/main/java/org/apache/amoro/maintainer/MaintainerMetrics.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.maintainer;
+
+/**
+ * Metrics collector interface for table maintenance operations. Implementations can collect metrics
+ * to different monitoring systems.
+ */
+public interface MaintainerMetrics {
+
+ /**
+ * Record orphan data files cleaning result.
+ *
+ * @param expected expected number of files to clean
+ * @param cleaned actual number of files cleaned
+ */
+ void recordOrphanDataFilesCleaned(int expected, int cleaned);
Review Comment:
The current solution is a temporary adaptation, as the monitoring function is not within the scope of this modification. This interface is designed to be compatible with **TableOrphanFilesCleaningMetrics**, and will be improved in subsequent pull requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "czy006 (via GitHub)" <gi...@apache.org>.
czy006 merged PR #4024:
URL: https://github.com/apache/amoro/pull/4024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "xxubai (via GitHub)" <gi...@apache.org>.
xxubai commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2660363749
##########
amoro-ams/src/main/java/org/apache/amoro/server/optimizing/maintainer/TableMaintainers.java:
##########
@@ -21,25 +21,39 @@
import org.apache.amoro.AmoroTable;
import org.apache.amoro.TableFormat;
import org.apache.amoro.TableRuntime;
+import org.apache.amoro.formats.iceberg.maintainer.MixedTableMaintainer;
+import org.apache.amoro.maintainer.TableMaintainer;
import org.apache.amoro.server.table.DefaultTableRuntime;
-import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
import org.apache.amoro.table.MixedTable;
import org.apache.iceberg.Table;
/** Factory for creating {@link TableMaintainer}. */
public class TableMaintainers {
- /** Create a {@link TableMaintainer} for the given table. */
+ /**
+ * Create a {@link TableMaintainer} for the given table.
+ *
+ * @deprecated Use {@link TableMaintainerFactory#create(AmoroTable, TableRuntime)} instead.
Review Comment:
Should we deprecate it in `0.10.0`?
##########
amoro-common/src/main/java/org/apache/amoro/maintainer/MaintainerMetrics.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.maintainer;
+
+/**
+ * Metrics collector interface for table maintenance operations. Implementations can collect metrics
+ * to different monitoring systems.
+ */
+public interface MaintainerMetrics {
+
+ /**
+ * Record orphan data files cleaning result.
+ *
+ * @param expected expected number of files to clean
+ * @param cleaned actual number of files cleaned
+ */
+ void recordOrphanDataFilesCleaned(int expected, int cleaned);
Review Comment:
I wonder whether this function could be made more generic. Since the Maintainer already supports other types of operations, should we also consider exposing interfaces for things like snapshot cleanup?
##########
amoro-common/src/main/java/org/apache/amoro/maintainer/MaintainerMetrics.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.amoro.maintainer;
+
+/**
+ * Metrics collector interface for table maintenance operations. Implementations can collect metrics
+ * to different monitoring systems.
+ */
+public interface MaintainerMetrics {
Review Comment:
```suggestion
public interface MetricsCollector {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "czy006 (via GitHub)" <gi...@apache.org>.
czy006 commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2654663983
##########
amoro-ams/src/test/java/org/apache/amoro/server/optimizing/maintainer/TestDataExpire.java:
##########
@@ -31,6 +31,8 @@
import org.apache.amoro.data.ChangeAction;
import org.apache.amoro.data.DataFileType;
import org.apache.amoro.data.PrimaryKeyedFile;
+import org.apache.amoro.formats.iceberg.maintainer.IcebergTableMaintainer;
Review Comment:
I tried my best, but it didn't work because it relies on other components such as **ExecutorBaseTest** and AMS, which have a large number of other dependencies within AMS. The executor of Maintainer is currently in AMS, and in the future we can implement an implementation based on external engine scheduling(next pr).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-4022] [Improvement]: AMS Iceberg maintainer moved to the amoro-iceberg module [amoro]
Posted by "xxubai (via GitHub)" <gi...@apache.org>.
xxubai commented on code in PR #4024:
URL: https://github.com/apache/amoro/pull/4024#discussion_r2660887608
##########
amoro-ams/src/main/java/org/apache/amoro/server/optimizing/maintainer/TableMaintainers.java:
##########
@@ -21,25 +21,39 @@
import org.apache.amoro.AmoroTable;
import org.apache.amoro.TableFormat;
import org.apache.amoro.TableRuntime;
+import org.apache.amoro.formats.iceberg.maintainer.MixedTableMaintainer;
+import org.apache.amoro.maintainer.TableMaintainer;
import org.apache.amoro.server.table.DefaultTableRuntime;
-import org.apache.amoro.shade.guava32.com.google.common.base.Preconditions;
import org.apache.amoro.table.MixedTable;
import org.apache.iceberg.Table;
/** Factory for creating {@link TableMaintainer}. */
public class TableMaintainers {
- /** Create a {@link TableMaintainer} for the given table. */
+ /**
+ * Create a {@link TableMaintainer} for the given table.
+ *
+ * @deprecated Use {@link TableMaintainerFactory#create(AmoroTable, TableRuntime)} instead.
Review Comment:
add a comment:
```
* @deprecated since 0.9.0, will be removed in 0.10.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org