You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "mlbiscoc (via GitHub)" <gi...@apache.org> on 2025/07/25 15:11:58 UTC
[PR] SOLR-17806: Migrate PeerSync and PeerSycWithLeader metrics to OTEL [solr]
mlbiscoc opened a new pull request, #3440:
URL: https://github.com/apache/solr/pull/3440
https://issues.apache.org/jira/browse/SOLR-17806
Migrate PeerSync and PeerSycWithLeader metrics and remove Dropwizard initializations.
```
# TYPE solr_core_leader_sync_errors_total counter
solr_core_leader_sync_errors_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard1_replica_n2",otel_scope_name="org.apache.solr",replica="replica_n2",shard="shard1"} 2.0
solr_core_leader_sync_errors_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard2_replica_n1",otel_scope_name="org.apache.solr",replica="replica_n1",shard="shard2"} 2.0
# HELP solr_core_leader_sync_skipped_total Total number of skipped syncs with leader
# TYPE solr_core_leader_sync_skipped_total counter
solr_core_leader_sync_skipped_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard1_replica_n2",otel_scope_name="org.apache.solr",replica="replica_n2",shard="shard1"} 1.0
solr_core_leader_sync_skipped_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard2_replica_n1",otel_scope_name="org.apache.solr",replica="replica_n1",shard="shard2"} 1.0
# HELP solr_core_leader_sync_time_milliseconds leader sync times
# TYPE solr_core_leader_sync_time_milliseconds histogram
solr_core_leader_sync_time_milliseconds_bucket{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard1_replica_n2",otel_scope_name="org.apache.solr",replica="replica_n2",shard="shard1",le="0.0"} 1
# TYPE solr_core_peer_sync_errors_total counter
solr_core_peer_sync_errors_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard1_replica_n4",otel_scope_name="org.apache.solr",replica="replica_n4",shard="shard1"} 2.0
solr_core_peer_sync_errors_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard2_replica_n6",otel_scope_name="org.apache.solr",replica="replica_n6",shard="shard2"} 2.0
# HELP solr_core_peer_sync_skipped_total Total number of skipped syncs with peer
# TYPE solr_core_peer_sync_skipped_total counter
solr_core_peer_sync_skipped_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard1_replica_n4",otel_scope_name="org.apache.solr",replica="replica_n4",shard="shard1"} 1.0
solr_core_peer_sync_skipped_total{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard2_replica_n6",otel_scope_name="org.apache.solr",replica="replica_n6",shard="shard2"} 1.0
# HELP solr_core_peer_sync_time_milliseconds Peer sync times
# TYPE solr_core_peer_sync_time_milliseconds histogram
solr_core_peer_sync_time_milliseconds_bucket{category="REPLICATION",collection="gettingstarted",core="gettingstarted_shard1_replica_n4",otel_scope_name="org.apache.solr",replica="replica_n4",shard="shard1",le="0.0"} 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17806: Migrate PeerSync and PeerSyncWithLeader metrics to OTEL [solr]
Posted by "mlbiscoc (via GitHub)" <gi...@apache.org>.
mlbiscoc commented on code in PR #3440:
URL: https://github.com/apache/solr/pull/3440#discussion_r2231386381
##########
solr/core/src/java/org/apache/solr/update/PeerSync.java:
##########
@@ -143,14 +143,28 @@ public SolrMetricsContext getSolrMetricsContext() {
return solrMetricsContext;
}
- // TODO SOLR-17458: Migrate to Otel
@Override
public void initializeMetrics(
SolrMetricsContext parentContext, Attributes attributes, String scope) {
this.solrMetricsContext = parentContext.getChildContext(this);
- syncTime = solrMetricsContext.timer("time", scope, METRIC_SCOPE);
- syncErrors = solrMetricsContext.counter("errors", scope, METRIC_SCOPE);
- syncSkipped = solrMetricsContext.counter("skipped", scope, METRIC_SCOPE);
+ var baseAttributes =
+ attributes.toBuilder()
+ .put("category", SolrInfoBean.Category.REPLICATION.toString())
+ .build();
+ syncErrors =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_core_peer_sync_errors", "Total number of sync errors with peer"),
+ baseAttributes);
+ syncSkipped =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_core_peer_sync_skipped", "Total number of skipped syncs with peer"),
+ baseAttributes);
+ syncTime =
+ new AttributedLongTimer(
+ solrMetricsContext.longHistogram("solr_core_peer_sync_time", "Peer sync times", "ms"),
+ baseAttributes);
Review Comment:
Dropwizard threw all these metrics under the same bucket which was confusing. I split them up between syncing with peers and syncing with leader as separate metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17806: Migrate PeerSync and PeerSyncWithLeader metrics to OTEL [solr]
Posted by "mlbiscoc (via GitHub)" <gi...@apache.org>.
mlbiscoc merged PR #3440:
URL: https://github.com/apache/solr/pull/3440
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17806: Migrate PeerSync and PeerSyncWithLeader metrics to OTEL [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #3440:
URL: https://github.com/apache/solr/pull/3440#discussion_r2232018996
##########
solr/core/src/java/org/apache/solr/update/PeerSync.java:
##########
@@ -143,14 +143,28 @@ public SolrMetricsContext getSolrMetricsContext() {
return solrMetricsContext;
}
- // TODO SOLR-17458: Migrate to Otel
@Override
public void initializeMetrics(
SolrMetricsContext parentContext, Attributes attributes, String scope) {
this.solrMetricsContext = parentContext.getChildContext(this);
- syncTime = solrMetricsContext.timer("time", scope, METRIC_SCOPE);
- syncErrors = solrMetricsContext.counter("errors", scope, METRIC_SCOPE);
- syncSkipped = solrMetricsContext.counter("skipped", scope, METRIC_SCOPE);
+ var baseAttributes =
+ attributes.toBuilder()
+ .put("category", SolrInfoBean.Category.REPLICATION.toString())
+ .build();
+ syncErrors =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_core_peer_sync_errors", "Total number of sync errors with peer"),
+ baseAttributes);
+ syncSkipped =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_core_peer_sync_skipped", "Total number of skipped syncs with peer"),
+ baseAttributes);
+ syncTime =
+ new AttributedLongTimer(
+ solrMetricsContext.longHistogram("solr_core_peer_sync_time", "Peer sync times", "ms"),
+ baseAttributes);
Review Comment:
Isn't "peer sync" always done with a leader, thus it's the same? Or so I thought.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17806: Migrate PeerSync and PeerSyncWithLeader metrics to OTEL [solr]
Posted by "mlbiscoc (via GitHub)" <gi...@apache.org>.
mlbiscoc commented on code in PR #3440:
URL: https://github.com/apache/solr/pull/3440#discussion_r2237801045
##########
solr/core/src/java/org/apache/solr/update/PeerSync.java:
##########
@@ -143,14 +143,28 @@ public SolrMetricsContext getSolrMetricsContext() {
return solrMetricsContext;
}
- // TODO SOLR-17458: Migrate to Otel
@Override
public void initializeMetrics(
SolrMetricsContext parentContext, Attributes attributes, String scope) {
this.solrMetricsContext = parentContext.getChildContext(this);
- syncTime = solrMetricsContext.timer("time", scope, METRIC_SCOPE);
- syncErrors = solrMetricsContext.counter("errors", scope, METRIC_SCOPE);
- syncSkipped = solrMetricsContext.counter("skipped", scope, METRIC_SCOPE);
+ var baseAttributes =
+ attributes.toBuilder()
+ .put("category", SolrInfoBean.Category.REPLICATION.toString())
+ .build();
+ syncErrors =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_core_peer_sync_errors", "Total number of sync errors with peer"),
+ baseAttributes);
+ syncSkipped =
+ new AttributedLongCounter(
+ solrMetricsContext.longCounter(
+ "solr_core_peer_sync_skipped", "Total number of skipped syncs with peer"),
+ baseAttributes);
+ syncTime =
+ new AttributedLongTimer(
+ solrMetricsContext.longHistogram("solr_core_peer_sync_time", "Peer sync times", "ms"),
+ baseAttributes);
Review Comment:
I believe most of the time peer sync is done with a leader majority of the time but in some cases like leader election the replicas do a [sync with each other](https://github.com/apache/solr/blob/6bb80bd72d5fa06035d10c157c27476fbb76188f/solr/core/src/java/org/apache/solr/cloud/SyncStrategy.java#L196) which probably triggers these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org