You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Alexander Lapin (Jira)" <ji...@apache.org> on 2022/11/04 17:06:00 UTC
[jira] [Updated] (IGNITE-18093) Introduce distribution zone manager for the purposes of zones configuration
[ https://issues.apache.org/jira/browse/IGNITE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Lapin updated IGNITE-18093:
-------------------------------------
Description:
h3. Motivation
Distribution zones manager is an ignite component that is responsible for both zones configuration(create/update/drop) and dataNodes calculation that is triggered by corresponding topology and zone modification events.
Within the scope of current ticket it's expected that DistributionZoneManager will be introduced with an internal API to create, drop and alter distribution zones.
h3. Definition of Done
* New ignite component DistributionZoneManager is introduced.
* It is inlined into ignite node start/stop flow.
* It's possible to create/alter/drop distribution zone that internally leads to cluster configuration changes.
h3. Implementation Notes
1. First of all we should introduce new root zones configuration schema:
{code:java}
@ConfigurationRoot(rootName = "zone", type = ConfigurationType.DISTRIBUTED)
public class DistributionZonesConfigurationSchema {
/** Global integer id counter. Used as an auto-increment counter to generate integer identifiers for distribution zone. */
@Value(hasDefault = true)
public int globalIdCounter = 0;
/** List of configured distribution zones. */
@NamedConfigValue
public DistributionZoneConfigurationSchema distributionZones;
}{code}
and zone configuration itself:
{code:java}
/**
* Distribution zone configuration schema class.
*/
@Config
public class DistributionZoneConfigurationSchema {
/** Zone name. */
@InjectedName
public String name;
/** Integer zone id. */
@Immutable
@Range(min = 1)
@Value(hasDefault = true)
public int zoneId = 1;
/** Partitions count. */
@Range(min = 0, max = 65_000)
@Value(hasDefault = true)
public int partitions = 25;
/** Count of partition replicas. */
@Range(min = 1)
@Value(hasDefault = true)
public int replicas = 1;
public int dataNodesAutoAdjustScaleUp;
public int dataNodesAutoAdjustScaleDown;
public int dataNodesAutoAdjust;
} {code}
Filter and affinity function type will be added to configuration later.
2. It's also required to introduce DistributionZoneManager as an IgniteComponent that should be started and stopped during node start and stop flow. The proper place to insert distributionZoneManager.start() is somewhere after clusterCfgMgr.start().
3. DistributionZoneManager should expose (as in internal API) methods to create alter and drop distribution zones.
4. Internally all that methods should call update configuration closures.
5. The only interesting part here is input data and drop validations.
* In case of drop we should check whether there are tables binded to given zone. Let's say that CASCADE is not supported for now.
* Zone name should be checked for uniqueness. Let's ask SE teem about proper way of doing such checks, maybe there's an annotation that will help.
* Neither scaleUp or scaleDown are compatible with common auto adjust, so we should complete create result future (yep it's async) exceptionally if user specifies both (scaleUp || scaleDown) and common auto adjust. In case of alter proposed property should override non-compatible ones.
* All-in-all some explicit validation, some configuration validators (see @TableValidator) and lot's of tests are expected.
6. Lot's of unit tests are expected to be written, that will check that distibutionZoneManager.create()/update()/drop() is either returns future completed exceptionally with proper exception or actually change the configuration.
was:
h3. Motivation
Distribution zones manager is an ignite component that is responsible for both zones configuration(create/update/drop) and dataNodes calculation that is triggered by corresponding topology and zone modification events.
Within the scope of current ticket it's expected that DistributionZoneManager will be introduced with an internal API to create, drop and alter distribution zones.
h3. Definition of Done
* New ignite component DistributionZoneManager is introduced.
* It is inlined into ignite node start/stop flow.
* It's possible to create/alter/drop distribution zone that internally leads to cluster configuration changes.
h3. Implementation Notes
1. First of all we should introduce new root zones configuration schema:
{code:java}
@ConfigurationRoot(rootName = "zone", type = ConfigurationType.DISTRIBUTED)
public class DistributionZonesConfigurationSchema {
/** Global integer id counter. Used as an auto-increment counter to generate integer identifiers for distribution zone. */
@Value(hasDefault = true)
public int globalIdCounter = 0;
/** List of configured distribution zones. */
@NamedConfigValue
public DistributionZoneConfigurationSchema distributionZones;
}{code}
and zone configuration itself:
{code:java}
/**
* Distribution zone configuration schema class.
*/
@Config
public class DistributionZoneConfigurationSchema {
/** Zone name. */
@InjectedName
public String name;
/** Integer zone id. */
@Immutable
@Range(min = 1)
@Value(hasDefault = true)
public int zoneId = 1;
/** Partitions count. */
@Range(min = 0, max = 65_000)
@Value(hasDefault = true)
public int partitions = 25;
/** Count of partition replicas. */
@Range(min = 1)
@Value(hasDefault = true)
public int replicas = 1;
public int dataNodesAutoAdjustScaleUp;
public int dataNodesAutoAdjustScaleDown;
public int dataNodesAutoAdjust;
} {code}
Filter and affinity function type will be added to configuration later.
2. It's also required to introduce DistributionZoneManager as an IgniteComponent that should be started and stopped during node start and stop flow. The proper place to insert distributionZoneManager.start() is somewhere after clusterCfgMgr.start().
3. DistributionZoneManager should expose (as in internal API) methods to create alter and drop distribution zones.
4. Internally all that methods should call update configuration closures.
5. The only interesting part here is input data and drop validations.
* In case of drop we should check whether there are tables binded to given zone. Let's say that CASCADE is not supported for now.
* Zone name should be checked for uniqueness. Let's ask SE teem about proper way of doing such checks, maybe there's an annotation that will help.
* Neither scaleUp or scaleDown are compatible with common auto adjust, so we should complete create result future (yep it's async) exceptionally if user specifies both (scaleUp || scaleDown) and common auto adjust. In case of alter proposed property should override non-compatible ones.
* All-in-all some explicit validation, some configuration validators (see @TableValidator) and lot's of tests are expected.
6. Lot's of unit tests are expected to be written, that will check that distibutionZoneManager.create()/update()/drop() is either returns future completed exceptionally with proper exception or actually change the configuration.
> Introduce distribution zone manager for the purposes of zones configuration
> ---------------------------------------------------------------------------
>
> Key: IGNITE-18093
> URL: https://issues.apache.org/jira/browse/IGNITE-18093
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Priority: Major
>
> h3. Motivation
> Distribution zones manager is an ignite component that is responsible for both zones configuration(create/update/drop) and dataNodes calculation that is triggered by corresponding topology and zone modification events.
> Within the scope of current ticket it's expected that DistributionZoneManager will be introduced with an internal API to create, drop and alter distribution zones.
> h3. Definition of Done
> * New ignite component DistributionZoneManager is introduced.
> * It is inlined into ignite node start/stop flow.
> * It's possible to create/alter/drop distribution zone that internally leads to cluster configuration changes.
> h3. Implementation Notes
> 1. First of all we should introduce new root zones configuration schema:
> {code:java}
> @ConfigurationRoot(rootName = "zone", type = ConfigurationType.DISTRIBUTED)
> public class DistributionZonesConfigurationSchema {
> /** Global integer id counter. Used as an auto-increment counter to generate integer identifiers for distribution zone. */
> @Value(hasDefault = true)
> public int globalIdCounter = 0;
>
> /** List of configured distribution zones. */
> @NamedConfigValue
> public DistributionZoneConfigurationSchema distributionZones;
> }{code}
> and zone configuration itself:
> {code:java}
> /**
> * Distribution zone configuration schema class.
> */
> @Config
> public class DistributionZoneConfigurationSchema {
> /** Zone name. */
> @InjectedName
> public String name;
> /** Integer zone id. */
> @Immutable
> @Range(min = 1)
> @Value(hasDefault = true)
> public int zoneId = 1;
> /** Partitions count. */
> @Range(min = 0, max = 65_000)
> @Value(hasDefault = true)
> public int partitions = 25;
> /** Count of partition replicas. */
> @Range(min = 1)
> @Value(hasDefault = true)
> public int replicas = 1;
> public int dataNodesAutoAdjustScaleUp;
> public int dataNodesAutoAdjustScaleDown;
> public int dataNodesAutoAdjust;
> } {code}
> Filter and affinity function type will be added to configuration later.
> 2. It's also required to introduce DistributionZoneManager as an IgniteComponent that should be started and stopped during node start and stop flow. The proper place to insert distributionZoneManager.start() is somewhere after clusterCfgMgr.start().
> 3. DistributionZoneManager should expose (as in internal API) methods to create alter and drop distribution zones.
> 4. Internally all that methods should call update configuration closures.
> 5. The only interesting part here is input data and drop validations.
> * In case of drop we should check whether there are tables binded to given zone. Let's say that CASCADE is not supported for now.
> * Zone name should be checked for uniqueness. Let's ask SE teem about proper way of doing such checks, maybe there's an annotation that will help.
> * Neither scaleUp or scaleDown are compatible with common auto adjust, so we should complete create result future (yep it's async) exceptionally if user specifies both (scaleUp || scaleDown) and common auto adjust. In case of alter proposed property should override non-compatible ones.
> * All-in-all some explicit validation, some configuration validators (see @TableValidator) and lot's of tests are expected.
> 6. Lot's of unit tests are expected to be written, that will check that distibutionZoneManager.create()/update()/drop() is either returns future completed exceptionally with proper exception or actually change the configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)