You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Arafat Khan (Jira)" <ji...@apache.org> on 2025/12/02 05:54:00 UTC
[jira] [Resolved] (HDDS-14046) ReconStorageContainerManagerFacade is not getting initialized properly after upgrade
[ https://issues.apache.org/jira/browse/HDDS-14046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arafat Khan resolved HDDS-14046.
--------------------------------
Resolution: Fixed
> ReconStorageContainerManagerFacade is not getting initialized properly after upgrade
> ------------------------------------------------------------------------------------
>
> Key: HDDS-14046
> URL: https://issues.apache.org/jira/browse/HDDS-14046
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Recon
> Reporter: Arafat Khan
> Assignee: Arafat Khan
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.2.0
>
>
> The issue is a race condition during the Recon startup sequence. The *{{ReconTaskStatusTableUpgradeAction}}* (which adds the *{{last_task_run_status}}* column) was being executed _after_ the Recon services were started.
> Specifically, *{{ReconServer.start()}}* initializes and registers {*}{{ReconTaskStatusMetrics}}{*}. The metrics system (or other background tasks started by {*}{{start()}}{*}) immediately attempts to query the *{{RECON_TASK_STATUS}}* table. Since the upgrade logic ({*}{{finalizeLayoutFeatures}}{*}) was scheduled to run _after_ {*}{{start()}}{*}, the query failed with *{{SQLSyntaxErrorException}}* because the new column did not exist yet.
> *The Error we are getting now -*
> {code:java}
> Caused by: java.sql.SQLSyntaxErrorException: Column 'RECON_TASK_STATUS.last_task_run_status' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'RECON_TASK_STATUS.last_task_run_status' is not a column in the target table. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.<init>(Unknown Source) at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source) at org.jooq.impl.ProviderEnabledConnection.prepareStatement(ProviderEnabledConnection.java:109) at org.jooq.impl.SettingsEnabledConnection.prepareStatement(SettingsEnabledConnection.java:73) at org.jooq.impl.AbstractResultQuery.prepare(AbstractResultQuery.java:239) at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:322) ... 49 moreCaused by: ERROR 42X04: Column 'RECON_TASK_STATUS.last_task_run_status' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'RECON_TASK_STATUS.last_task_run_status' is not a column in the target table. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.sql.compile.ColumnReference.bindExpression(Unknown Source) at org.apache.derby.impl.sql.compile.ColumnReference.bindExpression(Unknown Source) at org.apache.derby.impl.sql.compile.ResultColumn.bindExpression(Unknown Source) at org.apache.derby.impl.sql.compile.ResultColumnList.bindExpressions(Unknown Source) at org.apache.derby.impl.sql.compile.SelectNode.bindExpressions(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.bindExpressions(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source) at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source) ... 58 more2025-11-24 06:52:10,562 ERROR [main]-org.apache.hadoop.ozone.recon.ReconServer: ReconStorageContainerManagerFacade is not initialized properly. Setting health status to false and adding error code.2025-11-24 06:52:10,563 INFO [main]-org.apache.hadoop.ozone.recon.ReconContext: Update healthStatus of Recon from true to false.2025-11-24 06:52:10,563 ERROR [main]-org.apache.hadoop.ozone.recon.ReconServer: OzoneManagerServiceProvider is not initialized properly. Setting health status to false and adding error code.2025-11-24 06:52:10,563 INFO [main]-org.apache.hadoop.ozone.recon.ReconContext: Update healthStatus of Recon from false to false. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org