You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "ddanielr (via GitHub)" <gi...@apache.org> on 2024/02/27 18:33:33 UTC
[PR] Reduce compaction properties [accumulo]
ddanielr opened a new pull request, #4311:
URL: https://github.com/apache/accumulo/pull/4311
Removes the "meta" and "root" compaction services and replaces them with
a single "default" compaction service.
Adds a constant for referencing the "default" compaction service name.
Renamed the DefaultCompactionPlanner to RatioBasedCompactionPlanner.
closes: #4308
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1505065725
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -88,9 +91,14 @@ class InitialConfiguration {
initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "meta");
+ initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
+ DEFAULT_COMPACTION_SERVICE_NAME);
- // TODO configure initial fate table config.. probably needs compaction config??
+ // ELASTICITY_TODO configure initial fate table config
Review Comment:
I left the `TODO` because I wasn't sure if there were any other configuration items still needed for the fate table and didn't want it to be missed.
There seems to be a lot of settings for the root and meta tables. I wasn't sure with the introduction of the fate table if it was also getting the `initialRootMetaConf` contents set as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr merged PR #4311:
URL: https://github.com/apache/accumulo/pull/4311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1505033454
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -51,7 +53,8 @@ class InitialConfiguration {
this.siteConf = siteConf;
initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "root");
+ initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
Review Comment:
We may be able to set no compaction dispatcher properties here. If we set nothing then it would fall back to the default config for all tables which should dispatch to the default service. So maybe we can remove the puts for all compaction dispatcher props for all tables in this class.
We should open a follow on issue to address the props during upgrade if there is not something that covers that. May be better as part of a larger issue w/ a checklist of stuff related to compaction upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1504858785
##########
core/src/main/java/org/apache/accumulo/core/conf/Property.java:
##########
@@ -49,42 +51,25 @@ public enum Property {
"Both major and minor compaction properties can be included under this prefix.", "3.1.0"),
COMPACTION_SERVICE_PREFIX(COMPACTION_PREFIX + "service.", null, PropertyType.PREFIX,
"This prefix should be used to define all properties for the compaction services."
- + "See {% jlink -f org.apache.accumulo.core.spi.compaction.DefaultCompactionPlanner %}.\n"
+ + "See {% jlink -f org.apache.accumulo.core.spi.compaction.RatioBasedCompactionPlanner %}.\n"
+ "A new external compaction service would be defined like the following:\n"
+ "`compaction.service.newService.planner="
- + "\"org.apache.accumulo.core.spi.compaction.DefaultCompactionPlanner\".`\n"
+ + "\"org.apache.accumulo.core.spi.compaction.RatioBasedCompactionPlanner\".`\n"
+ "`compaction.service.newService.opts.groups=\""
+ "[{\"name\": \"small\", \"maxSize\":\"32M\"},"
Review Comment:
Should 'name' be replaced with 'group' in this example?
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -88,9 +91,14 @@ class InitialConfiguration {
initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "meta");
+ initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
+ DEFAULT_COMPACTION_SERVICE_NAME);
- // TODO configure initial fate table config.. probably needs compaction config??
+ // ELASTICITY_TODO configure initial fate table config
Review Comment:
Is this TODO still needed? It looks like you are configuring it below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1505084401
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -51,7 +53,8 @@ class InitialConfiguration {
this.siteConf = siteConf;
initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "root");
+ initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
Review Comment:
#4289 now validates properties as part of the upgrade, but I'm not sure how the compaction properties will be handled. #4273 will remove the need for additional properties to be set, but these dispatch opts will still work.
I'm fine with removing these entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1505982046
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -88,9 +91,14 @@ class InitialConfiguration {
initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "meta");
+ initialMetaConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
+ DEFAULT_COMPACTION_SERVICE_NAME);
- // TODO configure initial fate table config.. probably needs compaction config??
+ // ELASTICITY_TODO configure initial fate table config
Review Comment:
Leaving the TODO since we are removing the compaction properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1506065789
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -51,7 +53,8 @@ class InitialConfiguration {
this.siteConf = siteConf;
initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "root");
+ initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
Review Comment:
Removed the properties and created #4315 for keeping track of the compaction properties as part of upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Reduce compaction properties [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4311:
URL: https://github.com/apache/accumulo/pull/4311#discussion_r1505160846
##########
server/base/src/main/java/org/apache/accumulo/server/init/InitialConfiguration.java:
##########
@@ -51,7 +53,8 @@ class InitialConfiguration {
this.siteConf = siteConf;
initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER.getKey(),
SimpleCompactionDispatcher.class.getName());
- initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service", "root");
+ initialRootConf.put(Property.TABLE_COMPACTION_DISPATCHER_OPTS.getKey() + "service",
Review Comment:
> but I'm not sure how the compaction properties will be handled.
The prop settings in the metadata table may be syntactically correct and semantically incorrect. Not sure the upgrade will catch that, depends on how deep the validation goes.
> I'm fine with removing these entirely.
Thinking we should remove them if what they are set to is equivalent to the defaults. Seems less confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org