[jira] [Updated] (IGNITE-15403) Eliminate excessive storage directories creation for table partitions

["Alexander Lapin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexander Lapin updated IGNITE-15403:
-------------------------------------
    Description: 
It's requited to create and resolve directories for storage only if a local node hosts partition raft node:
{code:java}
if (nodes.stream().map(ClusterNode::name).collect(Collectors.toSet()).contains(locNodeName))
{code}
However, at this moment it's ignored and storage directory will be created on every node:
{code:java}
private void createTableLocally(
    String name,
    UUID tblId,
    List<List<ClusterNode>> assignment,
    SchemaRegistry schemaReg
) {
    int partitions = assignment.size();

    var partitionsGroupsFutures = new ArrayList<CompletableFuture<RaftGroupService>>();

    Path storageDir = partitionsStoreDir.resolve(name);

    try {
        Files.createDirectories(storageDir);
    } catch (IOException e) {
        throw new IgniteInternalException(
            "Failed to create partitions store directory for " + name + ": " + e.getMessage(),
            e
        );
    }

    for (int p = 0; p < partitions; p++) {
        RocksDbStorage storage = new RocksDbStorage(
            storageDir.resolve(String.valueOf(p)),
            ByteBuffer::compareTo
        );
{code}
In other words, storage directory is created regardless of whether a raft node is created or not.

 

  was:
It's requited to create and resolve directories for storage only if a local node hosts partition raft node:

 
{code:java}
if (nodes.stream().map(ClusterNode::name).collect(Collectors.toSet()).contains(locNodeName))
{code}
However, at this moment it's ignored and storage directory will be created on every node:

 

 
{code:java}
private void createTableLocally(
    String name,
    UUID tblId,
    List<List<ClusterNode>> assignment,
    SchemaRegistry schemaReg
) {
    int partitions = assignment.size();

    var partitionsGroupsFutures = new ArrayList<CompletableFuture<RaftGroupService>>();

    Path storageDir = partitionsStoreDir.resolve(name);

    try {
        Files.createDirectories(storageDir);
    } catch (IOException e) {
        throw new IgniteInternalException(
            "Failed to create partitions store directory for " + name + ": " + e.getMessage(),
            e
        );
    }

    for (int p = 0; p < partitions; p++) {
        RocksDbStorage storage = new RocksDbStorage(
            storageDir.resolve(String.valueOf(p)),
            ByteBuffer::compareTo
        );
{code}
In other words, storage directory is created regardless of whether a raft node is created or not.

 


> Eliminate excessive storage directories creation for table partitions
> ---------------------------------------------------------------------
>
>                 Key: IGNITE-15403
>                 URL: https://issues.apache.org/jira/browse/IGNITE-15403
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Alexander Lapin
>            Assignee: Alexander Lapin
>            Priority: Major
>              Labels: ignite-3
>
> It's requited to create and resolve directories for storage only if a local node hosts partition raft node:
> {code:java}
> if (nodes.stream().map(ClusterNode::name).collect(Collectors.toSet()).contains(locNodeName))
> {code}
> However, at this moment it's ignored and storage directory will be created on every node:
> {code:java}
> private void createTableLocally(
>     String name,
>     UUID tblId,
>     List<List<ClusterNode>> assignment,
>     SchemaRegistry schemaReg
> ) {
>     int partitions = assignment.size();
>     var partitionsGroupsFutures = new ArrayList<CompletableFuture<RaftGroupService>>();
>     Path storageDir = partitionsStoreDir.resolve(name);
>     try {
>         Files.createDirectories(storageDir);
>     } catch (IOException e) {
>         throw new IgniteInternalException(
>             "Failed to create partitions store directory for " + name + ": " + e.getMessage(),
>             e
>         );
>     }
>     for (int p = 0; p < partitions; p++) {
>         RocksDbStorage storage = new RocksDbStorage(
>             storageDir.resolve(String.valueOf(p)),
>             ByteBuffer::compareTo
>         );
> {code}
> In other words, storage directory is created regardless of whether a raft node is created or not.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)