You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Arafat Khan (Jira)" <ji...@apache.org> on 2025/12/02 05:54:00 UTC

[jira] [Resolved] (HDDS-14046) ReconStorageContainerManagerFacade is not getting initialized properly after upgrade

     [ https://issues.apache.org/jira/browse/HDDS-14046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arafat Khan resolved HDDS-14046.
--------------------------------
    Resolution: Fixed

> ReconStorageContainerManagerFacade is not getting initialized properly after upgrade
> ------------------------------------------------------------------------------------
>
>                 Key: HDDS-14046
>                 URL: https://issues.apache.org/jira/browse/HDDS-14046
>             Project: Apache Ozone
>          Issue Type: Improvement
>          Components: Ozone Recon
>            Reporter: Arafat Khan
>            Assignee: Arafat Khan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.2.0
>
>
> The issue is a race condition during the Recon startup sequence. The *{{ReconTaskStatusTableUpgradeAction}}* (which adds the *{{last_task_run_status}}* column) was being executed _after_ the Recon services were started.
> Specifically, *{{ReconServer.start()}}* initializes and registers {*}{{ReconTaskStatusMetrics}}{*}. The metrics system (or other background tasks started by {*}{{start()}}{*}) immediately attempts to query the *{{RECON_TASK_STATUS}}* table. Since the upgrade logic ({*}{{finalizeLayoutFeatures}}{*}) was scheduled to run _after_ {*}{{start()}}{*}, the query failed with *{{SQLSyntaxErrorException}}* because the new column did not exist yet.
> *The Error we are getting now -* 
> {code:java}
> Caused by: java.sql.SQLSyntaxErrorException: Column 'RECON_TASK_STATUS.last_task_run_status' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE  statement then 'RECON_TASK_STATUS.last_task_run_status' is not a column in the target table.	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)	at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.<init>(Unknown Source)	at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown Source)	at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)	at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)	at org.jooq.impl.ProviderEnabledConnection.prepareStatement(ProviderEnabledConnection.java:109)	at org.jooq.impl.SettingsEnabledConnection.prepareStatement(SettingsEnabledConnection.java:73)	at org.jooq.impl.AbstractResultQuery.prepare(AbstractResultQuery.java:239)	at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:322)	... 49 moreCaused by: ERROR 42X04: Column 'RECON_TASK_STATUS.last_task_run_status' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE  statement then 'RECON_TASK_STATUS.last_task_run_status' is not a column in the target table.	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)	at org.apache.derby.impl.sql.compile.ColumnReference.bindExpression(Unknown Source)	at org.apache.derby.impl.sql.compile.ColumnReference.bindExpression(Unknown Source)	at org.apache.derby.impl.sql.compile.ResultColumn.bindExpression(Unknown Source)	at org.apache.derby.impl.sql.compile.ResultColumnList.bindExpressions(Unknown Source)	at org.apache.derby.impl.sql.compile.SelectNode.bindExpressions(Unknown Source)	at org.apache.derby.impl.sql.compile.DMLStatementNode.bindExpressions(Unknown Source)	at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)	at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)	at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)	at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)	at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)	... 58 more2025-11-24 06:52:10,562 ERROR [main]-org.apache.hadoop.ozone.recon.ReconServer: ReconStorageContainerManagerFacade is not initialized properly. Setting health status to false and adding error code.2025-11-24 06:52:10,563 INFO [main]-org.apache.hadoop.ozone.recon.ReconContext: Update healthStatus of Recon from true to false.2025-11-24 06:52:10,563 ERROR [main]-org.apache.hadoop.ozone.recon.ReconServer: OzoneManagerServiceProvider is not initialized properly. Setting health status to false and adding error code.2025-11-24 06:52:10,563 INFO [main]-org.apache.hadoop.ozone.recon.ReconContext: Update healthStatus of Recon from false to false. {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org