[jira] [Resolved] (IMPALA-12666) Statestore topic metrics are wrong when impalads are shutdown

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang resolved IMPALA-12666.
-------------------------------------
    Fix Version/s: Impala 4.4.0
       Resolution: Fixed

> Statestore topic metrics are wrong when impalads are shutdown
> -------------------------------------------------------------
>
>                 Key: IMPALA-12666
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12666
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Major
>             Fix For: Impala 4.4.0
>
>
> While looking into the statestore codes, I found we incorrectly update the metrics in Statestore::Topic::DeleteIfVersionsMatch()
> {code:cpp}
> void Statestore::Topic::DeleteIfVersionsMatch(TopicEntry::Version version,
>     const Statestore::TopicEntryKey& key) {
>   // Acquire exclusive lock - we are modifying the topic.
>   lock_guard<shared_mutex> write_lock(lock_);
>   TopicEntryMap::iterator entry_it = entries_.find(key);
>   if (entry_it != entries_.end() && entry_it->second.version() == version) {
>     // Add a new entry with the the version history for this deletion and remove the old
>     // entry
>     topic_update_log_.erase(version);
>     topic_update_log_.emplace(++last_version_, key);
>     value_size_metric_->Increment(entry_it->second.value().size()); // We shouldn't bump this since nothing is added into the entrics_ map.
>     topic_size_metric_->Increment(entry_it->second.value().size()); // Same here.
>     entry_it->second.SetDeleted(true);
>     entry_it->second.SetVersion(last_version_);
>   }
> }{code}
> value_size_metric_ and topic_size_metric_ are shown in the "/metrics" page as "statestore.total-value-size-bytes" and "statestore.total-topic-size-bytes".
> Each topic has two counters: total_key_size_bytes_ and total_value_size_bytes_.
> value_size_metric_ tracks the sum of total_value_size_bytes_ across all topics. We shouldn't bump the metric here since we just mark a topic entry as deleted but not removing it. Note that in other places, we will always bump total_value_size_bytes_ and value_size_metric_. This is the only place that we just bump value_size_metric_ without total_value_size_bytes_ so it's wrong.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org