[jira] [Created] (HDDS-12228) Fix Duplicate Key Violation Condition in FileSizeCountTask

["Arafat Khan (Jira)" <ji...@apache.org>]

Arafat Khan created HDDS-12228:
----------------------------------

             Summary: Fix Duplicate Key Violation Condition in FileSizeCountTask
                 Key: HDDS-12228
                 URL: https://issues.apache.org/jira/browse/HDDS-12228
             Project: Apache Ozone
          Issue Type: Improvement
          Components: Ozone Recon
            Reporter: Arafat Khan
             Fix For: 2.0.0


In the *{{FileSizeCountTask}}* class, the update logic in {{*writeCountsToDB()*}} is never reached because *{{isDbTruncated}}* is always passed as {{{}true{}}}. This means that records are always inserted without checking for existing records, leading to SQL exceptions due to duplicate primary key constraints.

Example of SQL Exception:
{code:java}
2025-02-06 12:57:03 Caused by: org.jooq.exception.DataAccessException: SQL [insert into "FILE_COUNT_BY_SIZE" ("volume", "bucket", "file_size", "count") values (cast(? as varchar(32672)), cast(? as varchar(32672)), cast(? as bigint), cast(? as bigint))]; The statement was aborted because it would have caused a duplicate key value in a unique or primary key constraint or unique index identified by 'pk_volume_bucket_file_size' defined on 'FILE_COUNT_BY_SIZE'.
{code}
Root Cause: The update logic is inside the following condition:
{code:java}
if (!isDbTruncated) {
    // Get the current count from the database and update
    Record3<String, String, Long> recordToFind =
        dslContext.newRecord(
            FILE_COUNT_BY_SIZE.VOLUME,
            FILE_COUNT_BY_SIZE.BUCKET,
            FILE_COUNT_BY_SIZE.FILE_SIZE)
            .value1(key.volume)
            .value2(key.bucket)
            .value3(key.fileSizeUpperBound);
    FileCountBySize fileCountRecord =
        fileCountBySizeDao.findById(recordToFind);
    if (fileCountRecord == null && newRecord.getCount() > 0L) {
        insertToDb.add(newRecord);
    } else if (fileCountRecord != null) {
        newRecord.setCount(fileCountRecord.getCount() + fileSizeCountMap.get(key));
        updateInDb.add(newRecord);
    }
} {code}
However, since isDbTruncated is always true, this logic is never executed, and the update is skipped. Instead, all records are inserted directly, leading to duplicate key violations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org