[jira] [Resolved] (HDDS-13648) Update NSSummary rebuilding implementation to queue based approach

["Sumit Agrawal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sumit Agrawal resolved HDDS-13648.
----------------------------------
    Fix Version/s: 2.1.0
       Resolution: Fixed

> Update NSSummary rebuilding implementation to queue based approach
> ------------------------------------------------------------------
>
>                 Key: HDDS-13648
>                 URL: https://issues.apache.org/jira/browse/HDDS-13648
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: Ozone Recon
>            Reporter: Devesh Kumar Singh
>            Assignee: Devesh Kumar Singh
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.1.0
>
>
> Currently rebuilding of NSSummary data is based on multiple scenarios and being triggered across multiple code paths which might bring inconsistency. So to improve that, synchronized based approach was implemented so that one thread at a time should be running and building NSSummary, but now after staging and queue based implementation, this also should be updated to more simpler and consistent approach along with existing tasks executions.
> *And following additional fixes and improvements:*
>  # *NSSummaryTask.java:273 Reliability* The reprocess task might terminate prematurely if shutdown is slow. executorService.shutdown() is called without awaitTermination. Potentially misleading success logs if shutdown is interrupted. P3 L Add executorService.awaitTermination with a reasonable timeout after shutdown().
>  # *NSSummaryTask.java:253 Perf->Reliability* Slower reprocess execution due to thread starvation. Fixed-size thread pool (n=2) is smaller than the number of parallel tasks (n=3). Increased latency for full namespace reprocessing, delaying Recon's data availability.
>  # *NSSummaryTask.java:267 (Ignoring InterruptedException)*
> When a thread is interrupted during Future.get(), the InterruptedException is caught, but the thread's interrupted status is not restored.
>     
> {code:java}
> } catch (InterruptedException | ExecutionException ex) {        LOG.error("Error while reprocessing NSSummary table in Recon DB.", ex);        REBUILD_STATE.set(RebuildState.FAILED);        return buildTaskResult(false);        // Missing Thread.currentThread().interrupt();      }{code}
>  
>  # {*}NSSummaryTaskDbEventHandler.java:264{*}: On DB write failure, log the size of the failed batch. 
> {code:java}
> LOG.error("Unable to write Namespace Summary data in Recon DB. batchSize={}", nsSummaryMap.size(), e);{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org