You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "wcy-fdu (via GitHub)" <gi...@apache.org> on 2023/04/12 15:07:21 UTC

[GitHub] [incubator-opendal] wcy-fdu opened a new pull request, #1930: feat: add prometheus layer support

wcy-fdu opened a new pull request, #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930

   This PR adds prometheus later support, which use prometheus register to collect metrics. ([rust-prometheus](https://docs.rs/prometheus/latest/prometheus/))
   
   `Prometheus layer` share almost the same metrics with `metrics Layer`, but there is a little different. For metrics of size type(`write_size`), `metrics Layer` use `Counter` to monitor, this is a little weird to me.
   1. Counter is only incremental
   2. Usually, prometheus(or observe via grafana) pull metrcis by time interval. Give an example, if it collect metrics every 15s, and within 15s, a write interface was called multiple times, Counter can not accurately record the size of data written each time during the time interval, this is even more inaccurate when the data is jittery or abnormal.
   
   Thus, I use `Histogram` to monitor size type. If the bucket is set properly, the size metrics can be reflected more accurately through multiple sampling. 
   
   There are two ways to set this bucket of `Histogram`, one is `DEFAULT_BUCKETS+scale`, which pass a special bucket vec, but only up to ten segments, another one is `exponential bucket`, which we can set `start, factor, bucket_count`, and is more flexible. I choose the latter one,  just like TiKV and RisingWave.
   
   By exporting the data collected by prometheus to a certain port, we can use grafana to observe the dynamics changes of these metrcis. (These are not included in this PR.)
   
   There is an example for using prometheus layer
   ```rust
    use log::debug;
    use log::info;
    use opendal::services;
    use opendal::Operator;
    use opendal::Result;
   
    use opendal::layers::PrometheusLayer;
    use prometheus::Encoder;
   
    /// Visit [`opendal::services`] for more service related config.
    /// Visit [`opendal::Object`] for more object level APIs.
    #[tokio::main]
    async fn main() -> Result<()> {
        // Pick a builder and configure it.
        let builder = services::Memory::default();
   
        let op = Operator::new(builder)
            .expect("must init")
            .layer(PrometheusLayer)
            .finish();
        debug!("operator: {op:?}");
   
        // Write data into object test.
        op.write("test", "Hello, World!").await?;
        // Read data from object.
        let bs = op.read("test").await?;
        info!("content: {}", String::from_utf8_lossy(&bs));
   
        // Get object metadata.
        let meta = op.stat("test").await?;
        info!("meta: {:?}", meta);
   
        // Export prometheus metrics.
        let mut buffer = Vec::<u8>::new();
        let encoder = prometheus::TextEncoder::new();
        encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
        println!("## Prometheus Metrics");
        println!("{}", String::from_utf8(buffer.clone()).unwrap());
        Ok(())
    }
   ```
   
   Output
   ```
   ## Prometheus Metrics
   # HELP batch_request_counts Total times of batch be called
   # TYPE batch_request_counts counter
   batch_request_counts 0
   # HELP batch_request_latency batch letency
   # TYPE batch_request_latency histogram
   batch_request_latency_bucket{le="0.01"} 0
   batch_request_latency_bucket{le="0.02"} 0
   batch_request_latency_bucket{le="0.04"} 0
   batch_request_latency_bucket{le="0.08"} 0
   batch_request_latency_bucket{le="0.16"} 0
   batch_request_latency_bucket{le="0.32"} 0
   batch_request_latency_bucket{le="0.64"} 0
   batch_request_latency_bucket{le="1.28"} 0
   batch_request_latency_bucket{le="2.56"} 0
   batch_request_latency_bucket{le="5.12"} 0
   batch_request_latency_bucket{le="10.24"} 0
   batch_request_latency_bucket{le="20.48"} 0
   batch_request_latency_bucket{le="40.96"} 0
   batch_request_latency_bucket{le="81.92"} 0
   batch_request_latency_bucket{le="163.84"} 0
   batch_request_latency_bucket{le="327.68"} 0
   batch_request_latency_bucket{le="+Inf"} 0
   batch_request_latency_sum 0
   batch_request_latency_count 0
   # HELP blocking_create_request_counts Total times of blocking_create be called
   # TYPE blocking_create_request_counts counter
   blocking_create_request_counts 0
   # HELP blocking_create_request_latency blocking create letency
   # TYPE blocking_create_request_latency histogram
   blocking_create_request_latency_bucket{le="0.01"} 0
   blocking_create_request_latency_bucket{le="0.02"} 0
   blocking_create_request_latency_bucket{le="0.04"} 0
   blocking_create_request_latency_bucket{le="0.08"} 0
   blocking_create_request_latency_bucket{le="0.16"} 0
   blocking_create_request_latency_bucket{le="0.32"} 0
   blocking_create_request_latency_bucket{le="0.64"} 0
   blocking_create_request_latency_bucket{le="1.28"} 0
   blocking_create_request_latency_bucket{le="2.56"} 0
   blocking_create_request_latency_bucket{le="5.12"} 0
   blocking_create_request_latency_bucket{le="10.24"} 0
   blocking_create_request_latency_bucket{le="20.48"} 0
   blocking_create_request_latency_bucket{le="40.96"} 0
   blocking_create_request_latency_bucket{le="81.92"} 0
   blocking_create_request_latency_bucket{le="163.84"} 0
   blocking_create_request_latency_bucket{le="327.68"} 0
   blocking_create_request_latency_bucket{le="+Inf"} 0
   blocking_create_request_latency_sum 0
   blocking_create_request_latency_count 0
   # HELP blocking_delete_request_counts Total times of blocking_delete be called
   # TYPE blocking_delete_request_counts counter
   blocking_delete_request_counts 0
   # HELP blocking_delete_request_latency blocking_delete letency
   # TYPE blocking_delete_request_latency histogram
   blocking_delete_request_latency_bucket{le="0.01"} 0
   blocking_delete_request_latency_bucket{le="0.02"} 0
   blocking_delete_request_latency_bucket{le="0.04"} 0
   blocking_delete_request_latency_bucket{le="0.08"} 0
   blocking_delete_request_latency_bucket{le="0.16"} 0
   blocking_delete_request_latency_bucket{le="0.32"} 0
   blocking_delete_request_latency_bucket{le="0.64"} 0
   blocking_delete_request_latency_bucket{le="1.28"} 0
   blocking_delete_request_latency_bucket{le="2.56"} 0
   blocking_delete_request_latency_bucket{le="5.12"} 0
   blocking_delete_request_latency_bucket{le="10.24"} 0
   blocking_delete_request_latency_bucket{le="20.48"} 0
   blocking_delete_request_latency_bucket{le="40.96"} 0
   blocking_delete_request_latency_bucket{le="81.92"} 0
   blocking_delete_request_latency_bucket{le="163.84"} 0
   blocking_delete_request_latency_bucket{le="327.68"} 0
   blocking_delete_request_latency_bucket{le="+Inf"} 0
   blocking_delete_request_latency_sum 0
   blocking_delete_request_latency_count 0
   # HELP blocking_list_request_counts Total times of blocking_list be called
   # TYPE blocking_list_request_counts counter
   blocking_list_request_counts 0
   # HELP blocking_list_request_latency blocking_list letency
   # TYPE blocking_list_request_latency histogram
   blocking_list_request_latency_bucket{le="0.01"} 0
   blocking_list_request_latency_bucket{le="0.02"} 0
   blocking_list_request_latency_bucket{le="0.04"} 0
   blocking_list_request_latency_bucket{le="0.08"} 0
   blocking_list_request_latency_bucket{le="0.16"} 0
   blocking_list_request_latency_bucket{le="0.32"} 0
   blocking_list_request_latency_bucket{le="0.64"} 0
   blocking_list_request_latency_bucket{le="1.28"} 0
   blocking_list_request_latency_bucket{le="2.56"} 0
   blocking_list_request_latency_bucket{le="5.12"} 0
   blocking_list_request_latency_bucket{le="10.24"} 0
   blocking_list_request_latency_bucket{le="20.48"} 0
   blocking_list_request_latency_bucket{le="40.96"} 0
   blocking_list_request_latency_bucket{le="81.92"} 0
   blocking_list_request_latency_bucket{le="163.84"} 0
   blocking_list_request_latency_bucket{le="327.68"} 0
   blocking_list_request_latency_bucket{le="+Inf"} 0
   blocking_list_request_latency_sum 0
   blocking_list_request_latency_count 0
   # HELP blocking_read_request_counts Total times of blocking_read be called
   # TYPE blocking_read_request_counts counter
   blocking_read_request_counts 0
   # HELP blocking_read_request_latency Histogram of the time spent on blocking_reading
   # TYPE blocking_read_request_latency histogram
   blocking_read_request_latency_bucket{le="0.01"} 0
   blocking_read_request_latency_bucket{le="0.02"} 0
   blocking_read_request_latency_bucket{le="0.04"} 0
   blocking_read_request_latency_bucket{le="0.08"} 0
   blocking_read_request_latency_bucket{le="0.16"} 0
   blocking_read_request_latency_bucket{le="0.32"} 0
   blocking_read_request_latency_bucket{le="0.64"} 0
   blocking_read_request_latency_bucket{le="1.28"} 0
   blocking_read_request_latency_bucket{le="2.56"} 0
   blocking_read_request_latency_bucket{le="5.12"} 0
   blocking_read_request_latency_bucket{le="10.24"} 0
   blocking_read_request_latency_bucket{le="20.48"} 0
   blocking_read_request_latency_bucket{le="40.96"} 0
   blocking_read_request_latency_bucket{le="81.92"} 0
   blocking_read_request_latency_bucket{le="163.84"} 0
   blocking_read_request_latency_bucket{le="327.68"} 0
   blocking_read_request_latency_bucket{le="+Inf"} 0
   blocking_read_request_latency_sum 0
   blocking_read_request_latency_count 0
   # HELP blocking_read_size blocking_read size
   # TYPE blocking_read_size histogram
   blocking_read_size_bucket{le="0.01"} 0
   blocking_read_size_bucket{le="0.02"} 0
   blocking_read_size_bucket{le="0.04"} 0
   blocking_read_size_bucket{le="0.08"} 0
   blocking_read_size_bucket{le="0.16"} 0
   blocking_read_size_bucket{le="0.32"} 0
   blocking_read_size_bucket{le="0.64"} 0
   blocking_read_size_bucket{le="1.28"} 0
   blocking_read_size_bucket{le="2.56"} 0
   blocking_read_size_bucket{le="5.12"} 0
   blocking_read_size_bucket{le="10.24"} 0
   blocking_read_size_bucket{le="20.48"} 0
   blocking_read_size_bucket{le="40.96"} 0
   blocking_read_size_bucket{le="81.92"} 0
   blocking_read_size_bucket{le="163.84"} 0
   blocking_read_size_bucket{le="327.68"} 0
   blocking_read_size_bucket{le="+Inf"} 0
   blocking_read_size_sum 0
   blocking_read_size_count 0
   # HELP blocking_scan_request_counts Total times of blocking_scan be called
   # TYPE blocking_scan_request_counts counter
   blocking_scan_request_counts 0
   # HELP blocking_scan_request_latency blocking_scan letency
   # TYPE blocking_scan_request_latency histogram
   blocking_scan_request_latency_bucket{le="0.01"} 0
   blocking_scan_request_latency_bucket{le="0.02"} 0
   blocking_scan_request_latency_bucket{le="0.04"} 0
   blocking_scan_request_latency_bucket{le="0.08"} 0
   blocking_scan_request_latency_bucket{le="0.16"} 0
   blocking_scan_request_latency_bucket{le="0.32"} 0
   blocking_scan_request_latency_bucket{le="0.64"} 0
   blocking_scan_request_latency_bucket{le="1.28"} 0
   blocking_scan_request_latency_bucket{le="2.56"} 0
   blocking_scan_request_latency_bucket{le="5.12"} 0
   blocking_scan_request_latency_bucket{le="10.24"} 0
   blocking_scan_request_latency_bucket{le="20.48"} 0
   blocking_scan_request_latency_bucket{le="40.96"} 0
   blocking_scan_request_latency_bucket{le="81.92"} 0
   blocking_scan_request_latency_bucket{le="163.84"} 0
   blocking_scan_request_latency_bucket{le="327.68"} 0
   blocking_scan_request_latency_bucket{le="+Inf"} 0
   blocking_scan_request_latency_sum 0
   blocking_scan_request_latency_count 0
   # HELP blocking_stat_request_counts Total times of blocking_stat be called
   # TYPE blocking_stat_request_counts counter
   blocking_stat_request_counts 0
   # HELP blocking_stat_request_latency blocking_stat letency
   # TYPE blocking_stat_request_latency histogram
   blocking_stat_request_latency_bucket{le="0.01"} 0
   blocking_stat_request_latency_bucket{le="0.02"} 0
   blocking_stat_request_latency_bucket{le="0.04"} 0
   blocking_stat_request_latency_bucket{le="0.08"} 0
   blocking_stat_request_latency_bucket{le="0.16"} 0
   blocking_stat_request_latency_bucket{le="0.32"} 0
   blocking_stat_request_latency_bucket{le="0.64"} 0
   blocking_stat_request_latency_bucket{le="1.28"} 0
   blocking_stat_request_latency_bucket{le="2.56"} 0
   blocking_stat_request_latency_bucket{le="5.12"} 0
   blocking_stat_request_latency_bucket{le="10.24"} 0
   blocking_stat_request_latency_bucket{le="20.48"} 0
   blocking_stat_request_latency_bucket{le="40.96"} 0
   blocking_stat_request_latency_bucket{le="81.92"} 0
   blocking_stat_request_latency_bucket{le="163.84"} 0
   blocking_stat_request_latency_bucket{le="327.68"} 0
   blocking_stat_request_latency_bucket{le="+Inf"} 0
   blocking_stat_request_latency_sum 0
   blocking_stat_request_latency_count 0
   # HELP blocking_write_request_counts Total times of blocking_write be called
   # TYPE blocking_write_request_counts counter
   blocking_write_request_counts 0
   # HELP blocking_write_request_latency Histogram of the time spent on blocking_writing
   # TYPE blocking_write_request_latency histogram
   blocking_write_request_latency_bucket{le="0.01"} 0
   blocking_write_request_latency_bucket{le="0.02"} 0
   blocking_write_request_latency_bucket{le="0.04"} 0
   blocking_write_request_latency_bucket{le="0.08"} 0
   blocking_write_request_latency_bucket{le="0.16"} 0
   blocking_write_request_latency_bucket{le="0.32"} 0
   blocking_write_request_latency_bucket{le="0.64"} 0
   blocking_write_request_latency_bucket{le="1.28"} 0
   blocking_write_request_latency_bucket{le="2.56"} 0
   blocking_write_request_latency_bucket{le="5.12"} 0
   blocking_write_request_latency_bucket{le="10.24"} 0
   blocking_write_request_latency_bucket{le="20.48"} 0
   blocking_write_request_latency_bucket{le="40.96"} 0
   blocking_write_request_latency_bucket{le="81.92"} 0
   blocking_write_request_latency_bucket{le="163.84"} 0
   blocking_write_request_latency_bucket{le="327.68"} 0
   blocking_write_request_latency_bucket{le="+Inf"} 0
   blocking_write_request_latency_sum 0
   blocking_write_request_latency_count 0
   # HELP blocking_write_size total size by blocking_write
   # TYPE blocking_write_size histogram
   blocking_write_size_bucket{le="0.01"} 0
   blocking_write_size_bucket{le="0.02"} 0
   blocking_write_size_bucket{le="0.04"} 0
   blocking_write_size_bucket{le="0.08"} 0
   blocking_write_size_bucket{le="0.16"} 0
   blocking_write_size_bucket{le="0.32"} 0
   blocking_write_size_bucket{le="0.64"} 0
   blocking_write_size_bucket{le="1.28"} 0
   blocking_write_size_bucket{le="2.56"} 0
   blocking_write_size_bucket{le="5.12"} 0
   blocking_write_size_bucket{le="10.24"} 0
   blocking_write_size_bucket{le="20.48"} 0
   blocking_write_size_bucket{le="40.96"} 0
   blocking_write_size_bucket{le="81.92"} 0
   blocking_write_size_bucket{le="163.84"} 0
   blocking_write_size_bucket{le="327.68"} 0
   blocking_write_size_bucket{le="+Inf"} 0
   blocking_write_size_sum 0
   blocking_write_size_count 0
   # HELP create_request_counts Total times of create be called
   # TYPE create_request_counts counter
   create_request_counts 0
   # HELP create_request_latency Histogram of the time spent on creating
   # TYPE create_request_latency histogram
   create_request_latency_bucket{le="0.01"} 0
   create_request_latency_bucket{le="0.02"} 0
   create_request_latency_bucket{le="0.04"} 0
   create_request_latency_bucket{le="0.08"} 0
   create_request_latency_bucket{le="0.16"} 0
   create_request_latency_bucket{le="0.32"} 0
   create_request_latency_bucket{le="0.64"} 0
   create_request_latency_bucket{le="1.28"} 0
   create_request_latency_bucket{le="2.56"} 0
   create_request_latency_bucket{le="5.12"} 0
   create_request_latency_bucket{le="10.24"} 0
   create_request_latency_bucket{le="20.48"} 0
   create_request_latency_bucket{le="40.96"} 0
   create_request_latency_bucket{le="81.92"} 0
   create_request_latency_bucket{le="163.84"} 0
   create_request_latency_bucket{le="327.68"} 0
   create_request_latency_bucket{le="+Inf"} 0
   create_request_latency_sum 0
   create_request_latency_count 0
   # HELP delete_request_counts Total times of delete be called
   # TYPE delete_request_counts counter
   delete_request_counts 0
   # HELP delete_request_latency delete letency
   # TYPE delete_request_latency histogram
   delete_request_latency_bucket{le="0.01"} 0
   delete_request_latency_bucket{le="0.02"} 0
   delete_request_latency_bucket{le="0.04"} 0
   delete_request_latency_bucket{le="0.08"} 0
   delete_request_latency_bucket{le="0.16"} 0
   delete_request_latency_bucket{le="0.32"} 0
   delete_request_latency_bucket{le="0.64"} 0
   delete_request_latency_bucket{le="1.28"} 0
   delete_request_latency_bucket{le="2.56"} 0
   delete_request_latency_bucket{le="5.12"} 0
   delete_request_latency_bucket{le="10.24"} 0
   delete_request_latency_bucket{le="20.48"} 0
   delete_request_latency_bucket{le="40.96"} 0
   delete_request_latency_bucket{le="81.92"} 0
   delete_request_latency_bucket{le="163.84"} 0
   delete_request_latency_bucket{le="327.68"} 0
   delete_request_latency_bucket{le="+Inf"} 0
   delete_request_latency_sum 0
   delete_request_latency_count 0
   # HELP list_request_counts Total times of list be called
   # TYPE list_request_counts counter
   list_request_counts 0
   # HELP list_request_latency list letency
   # TYPE list_request_latency histogram
   list_request_latency_bucket{le="0.01"} 0
   list_request_latency_bucket{le="0.02"} 0
   list_request_latency_bucket{le="0.04"} 0
   list_request_latency_bucket{le="0.08"} 0
   list_request_latency_bucket{le="0.16"} 0
   list_request_latency_bucket{le="0.32"} 0
   list_request_latency_bucket{le="0.64"} 0
   list_request_latency_bucket{le="1.28"} 0
   list_request_latency_bucket{le="2.56"} 0
   list_request_latency_bucket{le="5.12"} 0
   list_request_latency_bucket{le="10.24"} 0
   list_request_latency_bucket{le="20.48"} 0
   list_request_latency_bucket{le="40.96"} 0
   list_request_latency_bucket{le="81.92"} 0
   list_request_latency_bucket{le="163.84"} 0
   list_request_latency_bucket{le="327.68"} 0
   list_request_latency_bucket{le="+Inf"} 0
   list_request_latency_sum 0
   list_request_latency_count 0
   # HELP metadata_request_counts Total times of metadata be called
   # TYPE metadata_request_counts counter
   metadata_request_counts 1
   # HELP metadata_request_latency Histogram of the time spent on getting metadata
   # TYPE metadata_request_latency histogram
   metadata_request_latency_bucket{le="0.01"} 1
   metadata_request_latency_bucket{le="0.02"} 1
   metadata_request_latency_bucket{le="0.04"} 1
   metadata_request_latency_bucket{le="0.08"} 1
   metadata_request_latency_bucket{le="0.16"} 1
   metadata_request_latency_bucket{le="0.32"} 1
   metadata_request_latency_bucket{le="0.64"} 1
   metadata_request_latency_bucket{le="1.28"} 1
   metadata_request_latency_bucket{le="2.56"} 1
   metadata_request_latency_bucket{le="5.12"} 1
   metadata_request_latency_bucket{le="10.24"} 1
   metadata_request_latency_bucket{le="20.48"} 1
   metadata_request_latency_bucket{le="40.96"} 1
   metadata_request_latency_bucket{le="81.92"} 1
   metadata_request_latency_bucket{le="163.84"} 1
   metadata_request_latency_bucket{le="327.68"} 1
   metadata_request_latency_bucket{le="+Inf"} 1
   metadata_request_latency_sum 0.000014875
   metadata_request_latency_count 1
   # HELP presign_request_counts Total times of presign be called
   # TYPE presign_request_counts counter
   presign_request_counts 0
   # HELP presign_request_latency presign letency
   # TYPE presign_request_latency histogram
   presign_request_latency_bucket{le="0.01"} 0
   presign_request_latency_bucket{le="0.02"} 0
   presign_request_latency_bucket{le="0.04"} 0
   presign_request_latency_bucket{le="0.08"} 0
   presign_request_latency_bucket{le="0.16"} 0
   presign_request_latency_bucket{le="0.32"} 0
   presign_request_latency_bucket{le="0.64"} 0
   presign_request_latency_bucket{le="1.28"} 0
   presign_request_latency_bucket{le="2.56"} 0
   presign_request_latency_bucket{le="5.12"} 0
   presign_request_latency_bucket{le="10.24"} 0
   presign_request_latency_bucket{le="20.48"} 0
   presign_request_latency_bucket{le="40.96"} 0
   presign_request_latency_bucket{le="81.92"} 0
   presign_request_latency_bucket{le="163.84"} 0
   presign_request_latency_bucket{le="327.68"} 0
   presign_request_latency_bucket{le="+Inf"} 0
   presign_request_latency_sum 0
   presign_request_latency_count 0
   # HELP read_request_counts Total times of read be called
   # TYPE read_request_counts counter
   read_request_counts 1
   # HELP read_request_latency Histogram of the time spent on reading
   # TYPE read_request_latency histogram
   read_request_latency_bucket{le="0.01"} 1
   read_request_latency_bucket{le="0.02"} 1
   read_request_latency_bucket{le="0.04"} 1
   read_request_latency_bucket{le="0.08"} 1
   read_request_latency_bucket{le="0.16"} 1
   read_request_latency_bucket{le="0.32"} 1
   read_request_latency_bucket{le="0.64"} 1
   read_request_latency_bucket{le="1.28"} 1
   read_request_latency_bucket{le="2.56"} 1
   read_request_latency_bucket{le="5.12"} 1
   read_request_latency_bucket{le="10.24"} 1
   read_request_latency_bucket{le="20.48"} 1
   read_request_latency_bucket{le="40.96"} 1
   read_request_latency_bucket{le="81.92"} 1
   read_request_latency_bucket{le="163.84"} 1
   read_request_latency_bucket{le="327.68"} 1
   read_request_latency_bucket{le="+Inf"} 1
   read_request_latency_sum 0.00001475
   read_request_latency_count 1
   # HELP read_size read size
   # TYPE read_size histogram
   read_size_bucket{le="0.01"} 0
   read_size_bucket{le="0.02"} 0
   read_size_bucket{le="0.04"} 0
   read_size_bucket{le="0.08"} 0
   read_size_bucket{le="0.16"} 0
   read_size_bucket{le="0.32"} 0
   read_size_bucket{le="0.64"} 0
   read_size_bucket{le="1.28"} 0
   read_size_bucket{le="2.56"} 0
   read_size_bucket{le="5.12"} 0
   read_size_bucket{le="10.24"} 0
   read_size_bucket{le="20.48"} 2
   read_size_bucket{le="40.96"} 2
   read_size_bucket{le="81.92"} 2
   read_size_bucket{le="163.84"} 2
   read_size_bucket{le="327.68"} 2
   read_size_bucket{le="+Inf"} 2
   read_size_sum 26
   read_size_count 2
   # HELP scan_request_counts Total times of scan be called
   # TYPE scan_request_counts counter
   scan_request_counts 0
   # HELP scan_request_latency scan letency
   # TYPE scan_request_latency histogram
   scan_request_latency_bucket{le="0.01"} 0
   scan_request_latency_bucket{le="0.02"} 0
   scan_request_latency_bucket{le="0.04"} 0
   scan_request_latency_bucket{le="0.08"} 0
   scan_request_latency_bucket{le="0.16"} 0
   scan_request_latency_bucket{le="0.32"} 0
   scan_request_latency_bucket{le="0.64"} 0
   scan_request_latency_bucket{le="1.28"} 0
   scan_request_latency_bucket{le="2.56"} 0
   scan_request_latency_bucket{le="5.12"} 0
   scan_request_latency_bucket{le="10.24"} 0
   scan_request_latency_bucket{le="20.48"} 0
   scan_request_latency_bucket{le="40.96"} 0
   scan_request_latency_bucket{le="81.92"} 0
   scan_request_latency_bucket{le="163.84"} 0
   scan_request_latency_bucket{le="327.68"} 0
   scan_request_latency_bucket{le="+Inf"} 0
   scan_request_latency_sum 0
   scan_request_latency_count 0
   # HELP stat_request_counts Total times of stat be called
   # TYPE stat_request_counts counter
   stat_request_counts 1
   # HELP stat_request_latency stat letency
   # TYPE stat_request_latency histogram
   stat_request_latency_bucket{le="0.01"} 1
   stat_request_latency_bucket{le="0.02"} 1
   stat_request_latency_bucket{le="0.04"} 1
   stat_request_latency_bucket{le="0.08"} 1
   stat_request_latency_bucket{le="0.16"} 1
   stat_request_latency_bucket{le="0.32"} 1
   stat_request_latency_bucket{le="0.64"} 1
   stat_request_latency_bucket{le="1.28"} 1
   stat_request_latency_bucket{le="2.56"} 1
   stat_request_latency_bucket{le="5.12"} 1
   stat_request_latency_bucket{le="10.24"} 1
   stat_request_latency_bucket{le="20.48"} 1
   stat_request_latency_bucket{le="40.96"} 1
   stat_request_latency_bucket{le="81.92"} 1
   stat_request_latency_bucket{le="163.84"} 1
   stat_request_latency_bucket{le="327.68"} 1
   stat_request_latency_bucket{le="+Inf"} 1
   stat_request_latency_sum 0.000006125
   stat_request_latency_count 1
   # HELP write_request_counts Total times of write be called
   # TYPE write_request_counts counter
   write_request_counts 1
   # HELP write_request_latency Histogram of the time spent on writing
   # TYPE write_request_latency histogram
   write_request_latency_bucket{le="0.01"} 1
   write_request_latency_bucket{le="0.02"} 1
   write_request_latency_bucket{le="0.04"} 1
   write_request_latency_bucket{le="0.08"} 1
   write_request_latency_bucket{le="0.16"} 1
   write_request_latency_bucket{le="0.32"} 1
   write_request_latency_bucket{le="0.64"} 1
   write_request_latency_bucket{le="1.28"} 1
   write_request_latency_bucket{le="2.56"} 1
   write_request_latency_bucket{le="5.12"} 1
   write_request_latency_bucket{le="10.24"} 1
   write_request_latency_bucket{le="20.48"} 1
   write_request_latency_bucket{le="40.96"} 1
   write_request_latency_bucket{le="81.92"} 1
   write_request_latency_bucket{le="163.84"} 1
   write_request_latency_bucket{le="327.68"} 1
   write_request_latency_bucket{le="+Inf"} 1
   write_request_latency_sum 0.000048083
   write_request_latency_count 1
   # HELP write_size write size
   # TYPE write_size histogram
   write_size_bucket{le="0.01"} 0
   write_size_bucket{le="0.02"} 0
   write_size_bucket{le="0.04"} 0
   write_size_bucket{le="0.08"} 0
   write_size_bucket{le="0.16"} 0
   write_size_bucket{le="0.32"} 0
   write_size_bucket{le="0.64"} 0
   write_size_bucket{le="1.28"} 0
   write_size_bucket{le="2.56"} 0
   write_size_bucket{le="5.12"} 0
   write_size_bucket{le="10.24"} 0
   write_size_bucket{le="20.48"} 1
   write_size_bucket{le="40.96"} 1
   write_size_bucket{le="81.92"} 1
   write_size_bucket{le="163.84"} 1
   write_size_bucket{le="327.68"} 1
   write_size_bucket{le="+Inf"} 1
   write_size_sum 13
   write_size_count 1
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507922682

   > inaccurate
   
   Keep one metrics for `Counter` looks good to me, as it always record the accurate value. 
   For duration, I think exponential  bucket can also meet requirements, as the span will not be too large.
   For size, I think we need to set reasonable `start`, `factor`, `bucket_count`, it will be different in different workload.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] ClSlaid commented on pull request #1930: feat: add prometheus layer support

Posted by "ClSlaid (via GitHub)" <gi...@apache.org>.
ClSlaid commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1505456831

   Please update to be even with `main` branch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] suyanhanx commented on pull request #1930: feat: add prometheus layer support

Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1505483101

   Could you clean your branch? 
   Some of your actions may have introduced too many commits that should not be on your branch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1506997785

   Now will use `schema` as labelAfter adding label for prometheus:
   `opendal_requests_total_metadata{schema="memory"} 1`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165769114


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub opendal_requests_total_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub opendal_requests_total_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_read: HistogramVec,
+    pub opendal_bytes_total_read: HistogramVec,
+
+    // write
+    pub opendal_requests_total_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_write: HistogramVec,
+    pub opendal_bytes_total_write: HistogramVec,
+
+    // stat
+    pub opendal_requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub opendal_requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub opendal_requests_total_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub opendal_requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub opendal_requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub opendal_requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub opendal_requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub opendal_requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_read: HistogramVec,
+    pub opendal_bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub opendal_requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_write: HistogramVec,
+    pub opendal_bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub opendal_requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub opendal_requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub opendal_requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub opendal_requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let opendal_requests_total_metadata = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_metadata",
+            "Total times of metadata be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // create
+        let opendal_requests_total_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_create",
+            "Total times of create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_create",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // read
+        let opendal_requests_total_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_read",
+            "Total times of read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_read",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_read",
+            "read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // write
+        let opendal_requests_total_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_write",
+            "Total times of write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_write",
+            "Histogram of the time spent on writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_write",
+            "write size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // stat
+        let opendal_requests_total_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_stat",
+            "Total times of stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_stat",
+            "stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // delete
+        let opendal_requests_total_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_delete",
+            "Total times of delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_delete",
+            "delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // list
+        let opendal_requests_total_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_list",
+            "Total times of list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_list",
+            "list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // scan
+        let opendal_requests_total_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_scan",
+            "Total times of scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_scan",
+            "scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // presign
+        let opendal_requests_total_presign = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_presign",
+            "Total times of presign be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_presign",
+            "presign letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_presign =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // batch
+        let opendal_requests_total_batch = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_batch",
+            "Total times of batch be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_batch",
+            "batch letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_batch =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_create
+        let opendal_requests_total_blocking_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_create",
+            "Total times of blocking_create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_create",
+            "blocking create letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_read
+        let opendal_requests_total_blocking_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_read",
+            "Total times of blocking_read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_read",
+            "Histogram of the time spent on blocking_reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_read",
+            "blocking_read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_write
+        let opendal_requests_total_blocking_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_write",
+            "Total times of blocking_write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_write",
+            "Histogram of the time spent on blocking_writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_write",
+            "total size by blocking_write",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_stat
+        let opendal_requests_total_blocking_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_stat",
+            "Total times of blocking_stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_stat",
+            "blocking_stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_delete
+        let opendal_requests_total_blocking_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_delete",
+            "Total times of blocking_delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_delete",
+            "blocking_delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_list
+        let opendal_requests_total_blocking_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_list",
+            "Total times of blocking_list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_list",
+            "blocking_list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_scan
+        let opendal_requests_total_blocking_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_scan",
+            "Total times of blocking_scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_scan",

Review Comment:
   Is this OK? `requests_total_read{operation="read",scheme="memory"} 1`
   Notice that `metadata` is not included in `Operation`, so I just missed this one temporarily. You can let me add it as well if you want. 😄



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165794254


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata

Review Comment:
   Question: is there meta cache in opendal? If yes, metadata latency is meaningful. 🤔



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata

Review Comment:
   Question: is there meta cache in OpenDAL? If yes, metadata latency is meaningful. 🤔



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165721704


##########
core/src/raw/accessor.rs:
##########
@@ -398,6 +398,8 @@ impl Accessor for () {
             max_batch_operations: None,
             capabilities: None.into(),
             hints: None.into(),
+            #[cfg(feature = "layers-prometheus")]

Review Comment:
   I see. Thanks for your feedback!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165769114


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub opendal_requests_total_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub opendal_requests_total_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_read: HistogramVec,
+    pub opendal_bytes_total_read: HistogramVec,
+
+    // write
+    pub opendal_requests_total_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_write: HistogramVec,
+    pub opendal_bytes_total_write: HistogramVec,
+
+    // stat
+    pub opendal_requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub opendal_requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub opendal_requests_total_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub opendal_requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub opendal_requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub opendal_requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub opendal_requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub opendal_requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_read: HistogramVec,
+    pub opendal_bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub opendal_requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_write: HistogramVec,
+    pub opendal_bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub opendal_requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub opendal_requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub opendal_requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub opendal_requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let opendal_requests_total_metadata = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_metadata",
+            "Total times of metadata be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // create
+        let opendal_requests_total_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_create",
+            "Total times of create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_create",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // read
+        let opendal_requests_total_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_read",
+            "Total times of read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_read",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_read",
+            "read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // write
+        let opendal_requests_total_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_write",
+            "Total times of write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_write",
+            "Histogram of the time spent on writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_write",
+            "write size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // stat
+        let opendal_requests_total_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_stat",
+            "Total times of stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_stat",
+            "stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // delete
+        let opendal_requests_total_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_delete",
+            "Total times of delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_delete",
+            "delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // list
+        let opendal_requests_total_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_list",
+            "Total times of list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_list",
+            "list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // scan
+        let opendal_requests_total_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_scan",
+            "Total times of scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_scan",
+            "scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // presign
+        let opendal_requests_total_presign = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_presign",
+            "Total times of presign be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_presign",
+            "presign letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_presign =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // batch
+        let opendal_requests_total_batch = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_batch",
+            "Total times of batch be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_batch",
+            "batch letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_batch =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_create
+        let opendal_requests_total_blocking_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_create",
+            "Total times of blocking_create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_create",
+            "blocking create letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_read
+        let opendal_requests_total_blocking_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_read",
+            "Total times of blocking_read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_read",
+            "Histogram of the time spent on blocking_reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_read",
+            "blocking_read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_write
+        let opendal_requests_total_blocking_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_write",
+            "Total times of blocking_write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_write",
+            "Histogram of the time spent on blocking_writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_write",
+            "total size by blocking_write",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_stat
+        let opendal_requests_total_blocking_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_stat",
+            "Total times of blocking_stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_stat",
+            "blocking_stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_delete
+        let opendal_requests_total_blocking_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_delete",
+            "Total times of blocking_delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_delete",
+            "blocking_delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_list
+        let opendal_requests_total_blocking_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_list",
+            "Total times of blocking_list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_list",
+            "blocking_list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_scan
+        let opendal_requests_total_blocking_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_scan",
+            "Total times of blocking_scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_scan",

Review Comment:
   Is this OK? `requests_total_read{operation="read",scheme="memory"} 1`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165699565


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,

Review Comment:
   We don't need to keep the `opendal_` prefix internally, `requests_total_metadata` looks good to me.



##########
core/src/raw/accessor.rs:
##########
@@ -398,6 +398,8 @@ impl Accessor for () {
             max_batch_operations: None,
             capabilities: None.into(),
             hints: None.into(),
+            #[cfg(feature = "layers-prometheus")]

Review Comment:
   Oh, layer doesn't work in this way. Layer should never hack into `Accessor` directly.
   
   We should add a new function in `PrometheusLayer` called `with_registry`.
   
   So that users can use it this way:
   
   ```rust
   let _ = Operator::new(services::Memory::default())
       .expect("must init")
       .layer(PrometheusLayer::default().with_registry(registry))
       .finish();
   ```
   
   Then, we can build `PrometheusAccessor` by:
   
   ```rust
   impl<A: Accessor> Layer<A> for PrometheusLayer {
       type LayeredAccessor = PrometheusAccessor<A>;
   
       fn layer(&self, inner: A) -> Self::LayeredAccessor {
           PrometheusAccessor {
               inner,
               stats: Arc::new(PrometheusMetrics::new(self.registry.unwrap_or_default())),
               schema: schema.to_string(),
           }
       }
   }
   ```



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub opendal_requests_total_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub opendal_requests_total_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_read: HistogramVec,
+    pub opendal_bytes_total_read: HistogramVec,
+
+    // write
+    pub opendal_requests_total_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_write: HistogramVec,
+    pub opendal_bytes_total_write: HistogramVec,
+
+    // stat
+    pub opendal_requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub opendal_requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub opendal_requests_total_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub opendal_requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub opendal_requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub opendal_requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub opendal_requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub opendal_requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_read: HistogramVec,
+    pub opendal_bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub opendal_requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_write: HistogramVec,
+    pub opendal_bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub opendal_requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub opendal_requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub opendal_requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub opendal_requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let opendal_requests_total_metadata = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_metadata",
+            "Total times of metadata be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // create
+        let opendal_requests_total_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_create",
+            "Total times of create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_create",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // read
+        let opendal_requests_total_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_read",
+            "Total times of read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_read",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_read",
+            "read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // write
+        let opendal_requests_total_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_write",
+            "Total times of write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_write",
+            "Histogram of the time spent on writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_write",
+            "write size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // stat
+        let opendal_requests_total_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_stat",
+            "Total times of stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_stat",
+            "stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // delete
+        let opendal_requests_total_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_delete",
+            "Total times of delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_delete",
+            "delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // list
+        let opendal_requests_total_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_list",
+            "Total times of list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_list",
+            "list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // scan
+        let opendal_requests_total_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_scan",
+            "Total times of scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_scan",
+            "scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // presign
+        let opendal_requests_total_presign = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_presign",
+            "Total times of presign be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_presign",
+            "presign letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_presign =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // batch
+        let opendal_requests_total_batch = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_batch",
+            "Total times of batch be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_batch",
+            "batch letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_batch =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_create
+        let opendal_requests_total_blocking_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_create",
+            "Total times of blocking_create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_create",
+            "blocking create letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_read
+        let opendal_requests_total_blocking_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_read",
+            "Total times of blocking_read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_read",
+            "Histogram of the time spent on blocking_reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_read",
+            "blocking_read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_write
+        let opendal_requests_total_blocking_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_write",
+            "Total times of blocking_write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_write",
+            "Histogram of the time spent on blocking_writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_write",
+            "total size by blocking_write",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_stat
+        let opendal_requests_total_blocking_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_stat",
+            "Total times of blocking_stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_stat",
+            "blocking_stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_delete
+        let opendal_requests_total_blocking_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_delete",
+            "Total times of blocking_delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_delete",
+            "blocking_delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_list
+        let opendal_requests_total_blocking_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_list",
+            "Total times of blocking_list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_list",
+            "blocking_list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_scan
+        let opendal_requests_total_blocking_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_scan",
+            "Total times of blocking_scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_scan",

Review Comment:
   We should treat `op` as label too.



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub opendal_requests_total_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub opendal_requests_total_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_read: HistogramVec,
+    pub opendal_bytes_total_read: HistogramVec,
+
+    // write
+    pub opendal_requests_total_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_write: HistogramVec,
+    pub opendal_bytes_total_write: HistogramVec,
+
+    // stat
+    pub opendal_requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub opendal_requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub opendal_requests_total_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub opendal_requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub opendal_requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub opendal_requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub opendal_requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub opendal_requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_read: HistogramVec,
+    pub opendal_bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub opendal_requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_write: HistogramVec,
+    pub opendal_bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub opendal_requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub opendal_requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub opendal_requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub opendal_requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let opendal_requests_total_metadata = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_metadata",
+            "Total times of metadata be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // create
+        let opendal_requests_total_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_create",
+            "Total times of create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_create",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // read
+        let opendal_requests_total_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_read",
+            "Total times of read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_read",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_read",
+            "read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // write
+        let opendal_requests_total_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_write",
+            "Total times of write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_write",
+            "Histogram of the time spent on writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_write",
+            "write size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // stat
+        let opendal_requests_total_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_stat",
+            "Total times of stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_stat",
+            "stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // delete
+        let opendal_requests_total_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_delete",
+            "Total times of delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_delete",
+            "delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // list
+        let opendal_requests_total_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_list",
+            "Total times of list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_list",
+            "list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // scan
+        let opendal_requests_total_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_scan",
+            "Total times of scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_scan",
+            "scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // presign
+        let opendal_requests_total_presign = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_presign",
+            "Total times of presign be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_presign",
+            "presign letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_presign =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // batch
+        let opendal_requests_total_batch = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_batch",
+            "Total times of batch be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_batch",
+            "batch letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_batch =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_create
+        let opendal_requests_total_blocking_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_create",
+            "Total times of blocking_create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_create",
+            "blocking create letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_read
+        let opendal_requests_total_blocking_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_read",
+            "Total times of blocking_read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_read",
+            "Histogram of the time spent on blocking_reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_read",
+            "blocking_read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_write
+        let opendal_requests_total_blocking_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_write",
+            "Total times of blocking_write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_write",
+            "Histogram of the time spent on blocking_writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_write",
+            "total size by blocking_write",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_stat
+        let opendal_requests_total_blocking_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_stat",
+            "Total times of blocking_stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_stat",
+            "blocking_stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_delete
+        let opendal_requests_total_blocking_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_delete",
+            "Total times of blocking_delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_delete",
+            "blocking_delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_list
+        let opendal_requests_total_blocking_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_list",
+            "Total times of blocking_list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_list",
+            "blocking_list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_scan
+        let opendal_requests_total_blocking_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_scan",
+            "Total times of blocking_scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_scan",
+            "blocking_scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        Self {
+            opendal_requests_total_metadata,
+            opendal_requests_duration_seconds_metadata,
+
+            opendal_requests_total_create,
+            opendal_requests_duration_seconds_create,
+
+            opendal_requests_total_read,
+            opendal_requests_duration_seconds_read,
+            opendal_bytes_total_read,
+
+            opendal_requests_total_write,
+            opendal_requests_duration_seconds_write,
+            opendal_bytes_total_write,
+
+            opendal_requests_total_stat,
+            opendal_requests_duration_seconds_stat,
+
+            opendal_requests_total_delete,
+            opendal_requests_duration_seconds_delete,
+
+            opendal_requests_total_list,
+            opendal_requests_duration_seconds_list,
+
+            opendal_requests_total_scan,
+            opendal_requests_duration_seconds_scan,
+
+            opendal_requests_total_presign,
+            opendal_requests_duration_seconds_presign,
+
+            opendal_requests_total_batch,
+            opendal_requests_duration_seconds_batch,
+
+            opendal_requests_total_blocking_create,
+            opendal_requests_duration_seconds_blocking_create,
+
+            opendal_requests_total_blocking_read,
+            opendal_requests_duration_seconds_blocking_read,
+            opendal_bytes_total_blocking_read,
+
+            opendal_requests_total_blocking_write,
+            opendal_requests_duration_seconds_blocking_write,
+            opendal_bytes_total_blocking_write,
+
+            opendal_requests_total_blocking_stat,
+            opendal_requests_duration_seconds_blocking_stat,
+
+            opendal_requests_total_blocking_delete,
+            opendal_requests_duration_seconds_blocking_delete,
+
+            opendal_requests_total_blocking_list,
+            opendal_requests_duration_seconds_blocking_list,
+
+            opendal_requests_total_blocking_scan,
+            opendal_requests_duration_seconds_blocking_scan,
+        }
+    }
+
+    /// error handling is the cold path, so we will not init error counters
+    /// in advance.
+    #[inline]
+    fn increment_errors_total(&self, op: Operation, kind: ErrorKind) {
+        debug!(
+            "Prometheus statistics metrics error, operation {} error {}",
+            op.into_static(),
+            kind.into_static()
+        );
+    }
+}
+
+#[derive(Clone)]
+pub struct PrometheusAccessor<A: Accessor> {
+    inner: A,
+    stats: Arc<PrometheusMetrics>,
+    schema: String,
+}
+
+impl<A: Accessor> Debug for PrometheusAccessor<A> {
+    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
+        f.debug_struct("PrometheusAccessor")
+            .field("inner", &self.inner)
+            .finish_non_exhaustive()
+    }
+}
+
+#[async_trait]
+impl<A: Accessor> LayeredAccessor for PrometheusAccessor<A> {
+    type Inner = A;
+    type Reader = PrometheusMetricWrapper<A::Reader>;
+    type BlockingReader = PrometheusMetricWrapper<A::BlockingReader>;
+    type Writer = PrometheusMetricWrapper<A::Writer>;
+    type BlockingWriter = PrometheusMetricWrapper<A::BlockingWriter>;
+    type Pager = A::Pager;
+    type BlockingPager = A::BlockingPager;
+
+    fn inner(&self) -> &Self::Inner {
+        &self.inner
+    }
+
+    fn metadata(&self) -> AccessorInfo {
+        self.stats
+            .opendal_requests_total_metadata
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_metadata
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.info();
+        timer.observe_duration();
+
+        result
+    }
+
+    async fn create(&self, path: &str, args: OpCreate) -> Result<RpCreate> {
+        self.stats
+            .opendal_requests_total_create
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_create
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let create_res = self.inner.create(path, args).await;
+
+        timer.observe_duration();
+        create_res.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::Create, e.kind());
+            e
+        })
+    }
+
+    async fn read(&self, path: &str, args: OpRead) -> Result<(RpRead, Self::Reader)> {
+        self.stats
+            .opendal_requests_total_read
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_read
+            .with_label_values(&[&self.schema])
+            .start_timer();
+
+        let read_res = self
+            .inner
+            .read(path, args)
+            .map(|v| {
+                v.map(|(rp, r)| {
+                    self.stats
+                        .opendal_bytes_total_read
+                        .with_label_values(&[&self.schema])
+                        .observe(rp.metadata().content_length() as f64);
+                    (
+                        rp,
+                        PrometheusMetricWrapper::new(
+                            r,
+                            Operation::Read,
+                            self.stats.clone(),
+                            &self.schema,
+                        ),
+                    )
+                })
+            })
+            .await;
+        timer.observe_duration();
+        read_res.map_err(|e| {
+            self.stats.increment_errors_total(Operation::Read, e.kind());
+            e
+        })
+    }
+
+    async fn write(&self, path: &str, args: OpWrite) -> Result<(RpWrite, Self::Writer)> {
+        self.stats
+            .opendal_requests_total_write
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_write
+            .with_label_values(&[&self.schema])
+            .start_timer();
+
+        let write_res = self
+            .inner
+            .write(path, args)
+            .map(|v| {
+                v.map(|(rp, r)| {
+                    (
+                        rp,
+                        PrometheusMetricWrapper::new(
+                            r,
+                            Operation::Write,
+                            self.stats.clone(),
+                            &self.schema,
+                        ),
+                    )
+                })
+            })
+            .await;
+        timer.observe_duration();
+        write_res.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::Write, e.kind());
+            e
+        })
+    }
+
+    async fn stat(&self, path: &str, args: OpStat) -> Result<RpStat> {
+        self.stats
+            .opendal_requests_total_stat
+            .with_label_values(&[&self.schema])
+            .inc();
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_stat
+            .with_label_values(&[&self.schema])
+            .start_timer();
+
+        let stat_res = self
+            .inner
+            .stat(path, args)
+            .inspect_err(|e| {
+                self.stats.increment_errors_total(Operation::Stat, e.kind());
+            })
+            .await;
+        timer.observe_duration();
+        stat_res.map_err(|e| {
+            self.stats.increment_errors_total(Operation::Stat, e.kind());
+            e
+        })
+    }
+
+    async fn delete(&self, path: &str, args: OpDelete) -> Result<RpDelete> {
+        self.stats
+            .opendal_requests_total_delete
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_delete
+            .with_label_values(&[&self.schema])
+            .start_timer();
+
+        let delete_res = self.inner.delete(path, args).await;
+        timer.observe_duration();
+        delete_res.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::Delete, e.kind());
+            e
+        })
+    }
+
+    async fn list(&self, path: &str, args: OpList) -> Result<(RpList, Self::Pager)> {
+        self.stats
+            .opendal_requests_total_list
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_list
+            .with_label_values(&[&self.schema])
+            .start_timer();
+
+        let list_res = self.inner.list(path, args).await;
+
+        timer.observe_duration();
+        list_res.map_err(|e| {
+            self.stats.increment_errors_total(Operation::List, e.kind());
+            e
+        })
+    }
+
+    async fn scan(&self, path: &str, args: OpScan) -> Result<(RpScan, Self::Pager)> {
+        self.stats
+            .opendal_requests_total_scan
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_scan
+            .with_label_values(&[&self.schema])
+            .start_timer();
+
+        let scan_res = self.inner.scan(path, args).await;
+        timer.observe_duration();
+        scan_res.map_err(|e| {
+            self.stats.increment_errors_total(Operation::Scan, e.kind());
+            e
+        })
+    }
+
+    async fn batch(&self, args: OpBatch) -> Result<RpBatch> {
+        self.stats
+            .opendal_requests_total_batch
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_batch
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.batch(args).await;
+
+        timer.observe_duration();
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::Batch, e.kind());
+            e
+        })
+    }
+
+    async fn presign(&self, path: &str, args: OpPresign) -> Result<RpPresign> {
+        self.stats
+            .opendal_requests_total_presign
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_presign
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.presign(path, args).await;
+        timer.observe_duration();
+
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::Presign, e.kind());
+            e
+        })
+    }
+
+    fn blocking_create(&self, path: &str, args: OpCreate) -> Result<RpCreate> {
+        self.stats
+            .opendal_requests_total_blocking_create
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_create
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_create(path, args);
+
+        timer.observe_duration();
+
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingCreate, e.kind());
+            e
+        })
+    }
+
+    fn blocking_read(&self, path: &str, args: OpRead) -> Result<(RpRead, Self::BlockingReader)> {
+        self.stats
+            .opendal_requests_total_blocking_read
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_read
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_read(path, args).map(|(rp, r)| {
+            self.stats
+                .opendal_bytes_total_read
+                .with_label_values(&[&self.schema])
+                .observe(rp.metadata().content_length() as f64);
+            (
+                rp,
+                PrometheusMetricWrapper::new(
+                    r,
+                    Operation::BlockingRead,
+                    self.stats.clone(),
+                    &self.schema,
+                ),
+            )
+        });
+        timer.observe_duration();
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingRead, e.kind());
+            e
+        })
+    }
+
+    fn blocking_write(&self, path: &str, args: OpWrite) -> Result<(RpWrite, Self::BlockingWriter)> {
+        self.stats
+            .opendal_requests_total_blocking_write
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_write
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_write(path, args).map(|(rp, r)| {
+            (
+                rp,
+                PrometheusMetricWrapper::new(
+                    r,
+                    Operation::BlockingWrite,
+                    self.stats.clone(),
+                    &self.schema,
+                ),
+            )
+        });
+        timer.observe_duration();
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingWrite, e.kind());
+            e
+        })
+    }
+
+    fn blocking_stat(&self, path: &str, args: OpStat) -> Result<RpStat> {
+        self.stats
+            .opendal_requests_total_blocking_stat
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_stat
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_stat(path, args);
+        timer.observe_duration();
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingStat, e.kind());
+            e
+        })
+    }
+
+    fn blocking_delete(&self, path: &str, args: OpDelete) -> Result<RpDelete> {
+        self.stats
+            .opendal_requests_total_blocking_delete
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_delete
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_delete(path, args);
+        timer.observe_duration();
+
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingDelete, e.kind());
+            e
+        })
+    }
+
+    fn blocking_list(&self, path: &str, args: OpList) -> Result<(RpList, Self::BlockingPager)> {
+        self.stats
+            .opendal_requests_total_blocking_list
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_list
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_list(path, args);
+        timer.observe_duration();
+
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingList, e.kind());
+            e
+        })
+    }
+
+    fn blocking_scan(&self, path: &str, args: OpScan) -> Result<(RpScan, Self::BlockingPager)> {
+        self.stats
+            .opendal_requests_total_blocking_scan
+            .with_label_values(&[&self.schema])
+            .inc();
+
+        let timer = self
+            .stats
+            .opendal_requests_duration_seconds_blocking_scan
+            .with_label_values(&[&self.schema])
+            .start_timer();
+        let result = self.inner.blocking_scan(path, args);
+        timer.observe_duration();
+        result.map_err(|e| {
+            self.stats
+                .increment_errors_total(Operation::BlockingScan, e.kind());
+            e
+        })
+    }
+}
+
+pub struct PrometheusMetricWrapper<R> {
+    inner: R,
+
+    op: Operation,
+    stats: Arc<PrometheusMetrics>,
+    schema: String,

Review Comment:
   `scheme` instead of `schema`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164425438


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;

Review Comment:
   Good Idea! let's keep the default register only used in example or test, and pass the user register as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo merged pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165797028


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata

Review Comment:
   Oh, `metadata` in `Accessor` just returns static info like `scheme`, `bucket_name` and so on. No real IO or syscall happened here. That's why it's not included in `Operation`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164427692


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let registry = prometheus::default_registry();
+
+        PrometheusMetricsAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub metadata_request_counts: GenericCounter<AtomicU64>,
+    pub metadata_request_latency: Histogram,
+
+    // create
+    pub create_request_counts: GenericCounter<AtomicU64>,
+    pub create_request_latency: Histogram,
+
+    /// read
+    pub read_request_counts: GenericCounter<AtomicU64>,
+    pub read_request_latency: Histogram,
+    pub read_size: Histogram,
+
+    // write
+    pub write_request_counts: GenericCounter<AtomicU64>,
+    pub write_request_latency: Histogram,
+    pub write_size: Histogram,
+
+    // stat
+    pub stat_request_counts: GenericCounter<AtomicU64>,
+    pub stat_request_latency: Histogram,
+
+    // delete
+    pub delete_request_counts: GenericCounter<AtomicU64>,
+    pub delete_request_latency: Histogram,
+
+    // list
+    pub list_request_counts: GenericCounter<AtomicU64>,
+    pub list_request_latency: Histogram,
+
+    // scan
+    pub scan_request_counts: GenericCounter<AtomicU64>,
+    pub scan_request_latency: Histogram,
+
+    // presign
+    pub presign_request_counts: GenericCounter<AtomicU64>,
+    pub presign_request_latency: Histogram,
+
+    // batch
+    pub batch_request_counts: GenericCounter<AtomicU64>,
+    pub batch_request_latency: Histogram,
+
+    // blocking create
+    pub blocking_create_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_create_request_latency: Histogram,
+
+    // blocking read
+    pub blocking_read_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_read_request_latency: Histogram,
+    pub blocking_read_size: Histogram,
+
+    // blocking write
+    pub blocking_write_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_write_request_latency: Histogram,
+    pub blocking_write_size: Histogram,
+
+    // blocking stat
+    pub blocking_stat_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_stat_request_latency: Histogram,
+
+    // blocking delete
+    pub blocking_delete_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_delete_request_latency: Histogram,
+
+    // blocking list
+    pub blocking_list_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_list_request_latency: Histogram,
+
+    // blocking scan
+    pub blocking_scan_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_scan_request_latency: Histogram,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let metadata_request_counts = register_int_counter_with_registry!(
+            "metadata_request_counts",

Review Comment:
   >does prometheus have the silimar concept of label?
   
   Sure, [an example](https://github.com/risingwavelabs/risingwave/blob/ee6d44f3dfb3a6a36de77033f2f7147bf77d1018/src/stream/src/executor/monitor/streaming_stats.rs#L104).
   
   Will refactor it soon.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507919774

   > > Do you mean only remain three basic metrics(`requests_total`, `requests_duration_seconds`,`bytes_total`), and use label to distinguish different requests?
   > 
   > Yes.
   
   This may cause inaccurate collection of some metrics, especially `HistogramVec`(`Histogram`). Different metrics have different value ranges, but if the share the same bucket, it will affect sampling. For example(not target the metrics in OpenDAL), size range of metrics A is `0~1KB`, and metrics B is `10Mb~100Mb`, not sure in this case whether `exponential  bucket` can satisfy.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507325587

   One last thing: All metrics should follow the same style, and move operation suffix (read, write, stat, ..) to label instead.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1508013952

   A quick test on RisingWave, passing the globally unique registry in RW, and monitor OpenDAL metrics in grafana:
   <img width="1501" alt="image" src="https://user-images.githubusercontent.com/58715567/231966213-778ab99d-fd1e-4691-a34a-51c4f3b0ac9b.png">
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165557560


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let registry = prometheus::default_registry();
+
+        PrometheusMetricsAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub metadata_request_counts: GenericCounter<AtomicU64>,
+    pub metadata_request_latency: Histogram,
+
+    // create
+    pub create_request_counts: GenericCounter<AtomicU64>,
+    pub create_request_latency: Histogram,
+
+    /// read
+    pub read_request_counts: GenericCounter<AtomicU64>,
+    pub read_request_latency: Histogram,
+    pub read_size: Histogram,
+
+    // write
+    pub write_request_counts: GenericCounter<AtomicU64>,
+    pub write_request_latency: Histogram,
+    pub write_size: Histogram,
+
+    // stat
+    pub stat_request_counts: GenericCounter<AtomicU64>,
+    pub stat_request_latency: Histogram,
+
+    // delete
+    pub delete_request_counts: GenericCounter<AtomicU64>,
+    pub delete_request_latency: Histogram,
+
+    // list
+    pub list_request_counts: GenericCounter<AtomicU64>,
+    pub list_request_latency: Histogram,
+
+    // scan
+    pub scan_request_counts: GenericCounter<AtomicU64>,
+    pub scan_request_latency: Histogram,
+
+    // presign
+    pub presign_request_counts: GenericCounter<AtomicU64>,
+    pub presign_request_latency: Histogram,
+
+    // batch
+    pub batch_request_counts: GenericCounter<AtomicU64>,
+    pub batch_request_latency: Histogram,
+
+    // blocking create
+    pub blocking_create_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_create_request_latency: Histogram,
+
+    // blocking read
+    pub blocking_read_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_read_request_latency: Histogram,
+    pub blocking_read_size: Histogram,
+
+    // blocking write
+    pub blocking_write_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_write_request_latency: Histogram,
+    pub blocking_write_size: Histogram,
+
+    // blocking stat
+    pub blocking_stat_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_stat_request_latency: Histogram,
+
+    // blocking delete
+    pub blocking_delete_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_delete_request_latency: Histogram,
+
+    // blocking list
+    pub blocking_list_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_list_request_latency: Histogram,
+
+    // blocking scan
+    pub blocking_scan_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_scan_request_latency: Histogram,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let metadata_request_counts = register_int_counter_with_registry!(
+            "metadata_request_counts",
+            "Total times of metadata be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "metadata_request_latency",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let metadata_request_latency = register_histogram_with_registry!(opts, registry).unwrap();
+
+        // create
+        let create_request_counts = register_int_counter_with_registry!(
+            "create_request_counts",
+            "Total times of create be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "create_request_latency",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let create_request_latency = register_histogram_with_registry!(opts, registry).unwrap();
+
+        // read
+        let read_request_counts = register_int_counter_with_registry!(
+            "read_request_counts",
+            "Total times of read be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "read_request_latency",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let read_request_latency = register_histogram_with_registry!(opts, registry).unwrap();
+
+        let opts = histogram_opts!(

Review Comment:
   From my personal perspective, I prefer to use `HistogramVec`(`Histogram`) to monitor size, for the reason, you can refer to the pr description.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507925532

   I am not an expert on Prometheus, but according to the document at https://prometheus.io/docs/practices/naming/#labels, it is recommended not to include labels in metric names.
   
   > * For duration, I think exponential  bucket can also meet requirements, as the span will not be too large
   >   (At most, it is the latency difference between accessing memory and accessing remote storage).
   
   Based on current API: 
   
   
   ```rust
   let opts = histogram_opts!(
       "bytes_total_blocking_write",
       "total size by blocking_write",
       exponential_buckets(0.01, 2.0, 16).unwrap()
   );
   let bytes_total_blocking_write =
       register_histogram_vec_with_registry!(opts, &["scheme", "operation"], registry)
           .unwrap();
   ```
   
   I thought we should use different histogram bucket for different labels?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507926542

   > 
   
   Ohhh, I see. Let me change it later.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] suyanhanx commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "suyanhanx (via GitHub)" <gi...@apache.org>.
suyanhanx commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164316230


##########
core/Cargo.toml:
##########


Review Comment:
   Maybe you can also update feature flags together?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507942273

   > I am not an expert on Prometheus, but according to the document at https://prometheus.io/docs/practices/naming/#labels, it is recommended not to include labels in metric names.
   > 
   > > * For duration, I think exponential  bucket can also meet requirements, as the span will not be too large
   > >   (At most, it is the latency difference between accessing memory and accessing remote storage).
   > 
   > Based on current API:
   > 
   > ```rust
   > let opts = histogram_opts!(
   >     "bytes_total_blocking_write",
   >     "total size by blocking_write",
   >     exponential_buckets(0.01, 2.0, 16).unwrap()
   > );
   > let bytes_total_blocking_write =
   >     register_histogram_vec_with_registry!(opts, &["scheme", "operation"], registry)
   >         .unwrap();
   > ```
   > 
   > I thought we can use different histogram bucket for different labels?
   
   Thanks for suggestion! Fixed, and it looks much clearer:
   ```
   requests_total{operation="read",scheme="memory"} 1
   requests_total{operation="stat",scheme="memory"} 1
   requests_total{operation="write",scheme="memory"} 1
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] ClSlaid commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "ClSlaid (via GitHub)" <gi...@apache.org>.
ClSlaid commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164281609


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,944 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use crate::ops::*;
+use crate::raw::*;
+use crate::*;
+
+use prometheus::{
+    core::{AtomicU64, GenericCounter},
+    register_int_counter_with_registry, Histogram,
+};
+use prometheus::{exponential_buckets, histogram_opts, register_histogram_with_registry, Registry};
+
+use crate::raw::Accessor;

Review Comment:
   Please group external crates with external crates, and in-crate imports with in-crate imports.
   
   Please do list imported items instead of grouping them in braces.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164404316


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;

Review Comment:
   How about allow user to input their own registry? As requested by https://github.com/apache/incubator-opendal/issues/1108#issuecomment-1367728906



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let registry = prometheus::default_registry();
+
+        PrometheusMetricsAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub metadata_request_counts: GenericCounter<AtomicU64>,
+    pub metadata_request_latency: Histogram,
+
+    // create
+    pub create_request_counts: GenericCounter<AtomicU64>,
+    pub create_request_latency: Histogram,
+
+    /// read
+    pub read_request_counts: GenericCounter<AtomicU64>,
+    pub read_request_latency: Histogram,
+    pub read_size: Histogram,
+
+    // write
+    pub write_request_counts: GenericCounter<AtomicU64>,
+    pub write_request_latency: Histogram,
+    pub write_size: Histogram,
+
+    // stat
+    pub stat_request_counts: GenericCounter<AtomicU64>,
+    pub stat_request_latency: Histogram,
+
+    // delete
+    pub delete_request_counts: GenericCounter<AtomicU64>,
+    pub delete_request_latency: Histogram,
+
+    // list
+    pub list_request_counts: GenericCounter<AtomicU64>,
+    pub list_request_latency: Histogram,
+
+    // scan
+    pub scan_request_counts: GenericCounter<AtomicU64>,
+    pub scan_request_latency: Histogram,
+
+    // presign
+    pub presign_request_counts: GenericCounter<AtomicU64>,
+    pub presign_request_latency: Histogram,
+
+    // batch
+    pub batch_request_counts: GenericCounter<AtomicU64>,
+    pub batch_request_latency: Histogram,
+
+    // blocking create
+    pub blocking_create_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_create_request_latency: Histogram,
+
+    // blocking read
+    pub blocking_read_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_read_request_latency: Histogram,
+    pub blocking_read_size: Histogram,
+
+    // blocking write
+    pub blocking_write_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_write_request_latency: Histogram,
+    pub blocking_write_size: Histogram,
+
+    // blocking stat
+    pub blocking_stat_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_stat_request_latency: Histogram,
+
+    // blocking delete
+    pub blocking_delete_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_delete_request_latency: Histogram,
+
+    // blocking list
+    pub blocking_list_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_list_request_latency: Histogram,
+
+    // blocking scan
+    pub blocking_scan_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_scan_request_latency: Histogram,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let metadata_request_counts = register_int_counter_with_registry!(
+            "metadata_request_counts",
+            "Total times of metadata be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "metadata_request_latency",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let metadata_request_latency = register_histogram_with_registry!(opts, registry).unwrap();
+
+        // create
+        let create_request_counts = register_int_counter_with_registry!(
+            "create_request_counts",
+            "Total times of create be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "create_request_latency",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let create_request_latency = register_histogram_with_registry!(opts, registry).unwrap();
+
+        // read
+        let read_request_counts = register_int_counter_with_registry!(
+            "read_request_counts",
+            "Total times of read be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "read_request_latency",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let read_request_latency = register_histogram_with_registry!(opts, registry).unwrap();
+
+        let opts = histogram_opts!(

Review Comment:
   We used `Counter` to store size and users can calculate the other metrics by dividing `duration_seconds` or sum up to calculate the total size.



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let registry = prometheus::default_registry();
+
+        PrometheusMetricsAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub metadata_request_counts: GenericCounter<AtomicU64>,
+    pub metadata_request_latency: Histogram,
+
+    // create
+    pub create_request_counts: GenericCounter<AtomicU64>,
+    pub create_request_latency: Histogram,
+
+    /// read
+    pub read_request_counts: GenericCounter<AtomicU64>,
+    pub read_request_latency: Histogram,
+    pub read_size: Histogram,
+
+    // write
+    pub write_request_counts: GenericCounter<AtomicU64>,
+    pub write_request_latency: Histogram,
+    pub write_size: Histogram,
+
+    // stat
+    pub stat_request_counts: GenericCounter<AtomicU64>,
+    pub stat_request_latency: Histogram,
+
+    // delete
+    pub delete_request_counts: GenericCounter<AtomicU64>,
+    pub delete_request_latency: Histogram,
+
+    // list
+    pub list_request_counts: GenericCounter<AtomicU64>,
+    pub list_request_latency: Histogram,
+
+    // scan
+    pub scan_request_counts: GenericCounter<AtomicU64>,
+    pub scan_request_latency: Histogram,
+
+    // presign
+    pub presign_request_counts: GenericCounter<AtomicU64>,
+    pub presign_request_latency: Histogram,
+
+    // batch
+    pub batch_request_counts: GenericCounter<AtomicU64>,
+    pub batch_request_latency: Histogram,
+
+    // blocking create
+    pub blocking_create_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_create_request_latency: Histogram,
+
+    // blocking read
+    pub blocking_read_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_read_request_latency: Histogram,
+    pub blocking_read_size: Histogram,
+
+    // blocking write
+    pub blocking_write_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_write_request_latency: Histogram,
+    pub blocking_write_size: Histogram,
+
+    // blocking stat
+    pub blocking_stat_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_stat_request_latency: Histogram,
+
+    // blocking delete
+    pub blocking_delete_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_delete_request_latency: Histogram,
+
+    // blocking list
+    pub blocking_list_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_list_request_latency: Histogram,
+
+    // blocking scan
+    pub blocking_scan_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_scan_request_latency: Histogram,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let metadata_request_counts = register_int_counter_with_registry!(
+            "metadata_request_counts",
+            "Total times of metadata be called",
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "metadata_request_latency",

Review Comment:
   I prefer to following the naming style as described at https://prometheus.io/docs/practices/naming/
   
   - opendal_requests_total
   - opendal_requests_duration_seconds
   - opendal_bytes_total



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let registry = prometheus::default_registry();
+
+        PrometheusMetricsAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub metadata_request_counts: GenericCounter<AtomicU64>,
+    pub metadata_request_latency: Histogram,
+
+    // create
+    pub create_request_counts: GenericCounter<AtomicU64>,
+    pub create_request_latency: Histogram,
+
+    /// read
+    pub read_request_counts: GenericCounter<AtomicU64>,
+    pub read_request_latency: Histogram,
+    pub read_size: Histogram,
+
+    // write
+    pub write_request_counts: GenericCounter<AtomicU64>,
+    pub write_request_latency: Histogram,
+    pub write_size: Histogram,
+
+    // stat
+    pub stat_request_counts: GenericCounter<AtomicU64>,
+    pub stat_request_latency: Histogram,
+
+    // delete
+    pub delete_request_counts: GenericCounter<AtomicU64>,
+    pub delete_request_latency: Histogram,
+
+    // list
+    pub list_request_counts: GenericCounter<AtomicU64>,
+    pub list_request_latency: Histogram,
+
+    // scan
+    pub scan_request_counts: GenericCounter<AtomicU64>,
+    pub scan_request_latency: Histogram,
+
+    // presign
+    pub presign_request_counts: GenericCounter<AtomicU64>,
+    pub presign_request_latency: Histogram,
+
+    // batch
+    pub batch_request_counts: GenericCounter<AtomicU64>,
+    pub batch_request_latency: Histogram,
+
+    // blocking create
+    pub blocking_create_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_create_request_latency: Histogram,
+
+    // blocking read
+    pub blocking_read_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_read_request_latency: Histogram,
+    pub blocking_read_size: Histogram,
+
+    // blocking write
+    pub blocking_write_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_write_request_latency: Histogram,
+    pub blocking_write_size: Histogram,
+
+    // blocking stat
+    pub blocking_stat_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_stat_request_latency: Histogram,
+
+    // blocking delete
+    pub blocking_delete_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_delete_request_latency: Histogram,
+
+    // blocking list
+    pub blocking_list_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_list_request_latency: Histogram,
+
+    // blocking scan
+    pub blocking_scan_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_scan_request_latency: Histogram,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let metadata_request_counts = register_int_counter_with_registry!(
+            "metadata_request_counts",

Review Comment:
   How about keeping the similiar naming style like `opendal_requests_total`?
   
   And does prometheus have the silimar concept of `label`? How about keeping our metrics organized in different lable instead?



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;

Review Comment:
   How about using `PrometheusAccessor`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507916491

   > Do you mean only remain three basic metrics(`requests_total`, `requests_duration_seconds`,`bytes_total`), and use label to distinguish different requests?
   
   Yes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164414249


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,944 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use crate::ops::*;
+use crate::raw::*;
+use crate::*;
+
+use prometheus::{
+    core::{AtomicU64, GenericCounter},
+    register_int_counter_with_registry, Histogram,
+};
+use prometheus::{exponential_buckets, histogram_opts, register_histogram_with_registry, Registry};
+
+use crate::raw::Accessor;

Review Comment:
   Oh, please avoid reviewing code format related issues :heart: 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165798908


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata

Review Comment:
   I see. Thanks, fixed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165783975


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let requests_total_metadata = register_int_counter_vec_with_registry!(
+            "requests_total_metadata",
+            "Total times of metadata be called",
+            &["scheme"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["scheme"], registry).unwrap();
+
+        // create
+        let requests_total_create = register_int_counter_vec_with_registry!(
+            "requests_total_create",

Review Comment:
   I'm guessing we should use `requests_total` here?



##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let requests_total_metadata = register_int_counter_vec_with_registry!(
+            "requests_total_metadata",
+            "Total times of metadata be called",
+            &["scheme"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["scheme"], registry).unwrap();
+
+        // create
+        let requests_total_create = register_int_counter_vec_with_registry!(
+            "requests_total_create",
+            "Total times of create be called",
+            &["scheme", "operation"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "requests_duration_seconds_create",

Review Comment:
   Maybe `requests_duration_seconds`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165781757


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub opendal_requests_total_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub opendal_requests_total_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_read: HistogramVec,
+    pub opendal_bytes_total_read: HistogramVec,
+
+    // write
+    pub opendal_requests_total_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_write: HistogramVec,
+    pub opendal_bytes_total_write: HistogramVec,
+
+    // stat
+    pub opendal_requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub opendal_requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub opendal_requests_total_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub opendal_requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub opendal_requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub opendal_requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub opendal_requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub opendal_requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_read: HistogramVec,
+    pub opendal_bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub opendal_requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_write: HistogramVec,
+    pub opendal_bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub opendal_requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub opendal_requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub opendal_requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub opendal_requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let opendal_requests_total_metadata = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_metadata",
+            "Total times of metadata be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // create
+        let opendal_requests_total_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_create",
+            "Total times of create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_create",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // read
+        let opendal_requests_total_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_read",
+            "Total times of read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_read",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_read",
+            "read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // write
+        let opendal_requests_total_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_write",
+            "Total times of write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_write",
+            "Histogram of the time spent on writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_write",
+            "write size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // stat
+        let opendal_requests_total_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_stat",
+            "Total times of stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_stat",
+            "stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // delete
+        let opendal_requests_total_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_delete",
+            "Total times of delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_delete",
+            "delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // list
+        let opendal_requests_total_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_list",
+            "Total times of list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_list",
+            "list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // scan
+        let opendal_requests_total_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_scan",
+            "Total times of scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_scan",
+            "scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // presign
+        let opendal_requests_total_presign = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_presign",
+            "Total times of presign be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_presign",
+            "presign letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_presign =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // batch
+        let opendal_requests_total_batch = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_batch",
+            "Total times of batch be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_batch",
+            "batch letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_batch =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_create
+        let opendal_requests_total_blocking_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_create",
+            "Total times of blocking_create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_create",
+            "blocking create letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_read
+        let opendal_requests_total_blocking_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_read",
+            "Total times of blocking_read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_read",
+            "Histogram of the time spent on blocking_reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_read",
+            "blocking_read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_write
+        let opendal_requests_total_blocking_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_write",
+            "Total times of blocking_write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_write",
+            "Histogram of the time spent on blocking_writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_write",
+            "total size by blocking_write",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_stat
+        let opendal_requests_total_blocking_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_stat",
+            "Total times of blocking_stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_stat",
+            "blocking_stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_delete
+        let opendal_requests_total_blocking_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_delete",
+            "Total times of blocking_delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_delete",
+            "blocking_delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_list
+        let opendal_requests_total_blocking_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_list",
+            "Total times of blocking_list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_list",
+            "blocking_list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_scan
+        let opendal_requests_total_blocking_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_scan",
+            "Total times of blocking_scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_scan",

Review Comment:
   > Notice that `metadata` is not included in `Operation`, so I just missed this one temporarily. You can let me add it as well if you want.
   
   metadata is opendal's own call and don't need metrics, so it's ok.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165782537


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata

Review Comment:
   We can remove this. Track metrics for metadata doesn't make sense :rofl: 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164425438


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;

Review Comment:
   Good Idea! let's keep the default register only used in example or test, and add pass the user register as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165769114


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1207 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let mut meta = inner.info();
+        let schema = meta.scheme();
+        let registry = match meta.prometheus_register() {
+            Some(register) => register,
+            None => prometheus::default_registry(),
+        };
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+            schema: schema.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub opendal_requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub opendal_requests_total_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub opendal_requests_total_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_read: HistogramVec,
+    pub opendal_bytes_total_read: HistogramVec,
+
+    // write
+    pub opendal_requests_total_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_write: HistogramVec,
+    pub opendal_bytes_total_write: HistogramVec,
+
+    // stat
+    pub opendal_requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub opendal_requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub opendal_requests_total_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub opendal_requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub opendal_requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub opendal_requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub opendal_requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub opendal_requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_read: HistogramVec,
+    pub opendal_bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub opendal_requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_write: HistogramVec,
+    pub opendal_bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub opendal_requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub opendal_requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub opendal_requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub opendal_requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub opendal_requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let opendal_requests_total_metadata = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_metadata",
+            "Total times of metadata be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_metadata",
+            "Histogram of the time spent on getting metadata",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_metadata =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // create
+        let opendal_requests_total_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_create",
+            "Total times of create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_create",
+            "Histogram of the time spent on creating",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // read
+        let opendal_requests_total_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_read",
+            "Total times of read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_read",
+            "Histogram of the time spent on reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_read",
+            "read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // write
+        let opendal_requests_total_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_write",
+            "Total times of write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_write",
+            "Histogram of the time spent on writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_write",
+            "write size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // stat
+        let opendal_requests_total_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_stat",
+            "Total times of stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_stat",
+            "stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // delete
+        let opendal_requests_total_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_delete",
+            "Total times of delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_delete",
+            "delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // list
+        let opendal_requests_total_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_list",
+            "Total times of list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_list",
+            "list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // scan
+        let opendal_requests_total_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_scan",
+            "Total times of scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_scan",
+            "scan letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_scan =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // presign
+        let opendal_requests_total_presign = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_presign",
+            "Total times of presign be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_presign",
+            "presign letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_presign =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // batch
+        let opendal_requests_total_batch = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_batch",
+            "Total times of batch be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_batch",
+            "batch letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_batch =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_create
+        let opendal_requests_total_blocking_create = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_create",
+            "Total times of blocking_create be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_create",
+            "blocking create letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_create =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_read
+        let opendal_requests_total_blocking_read = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_read",
+            "Total times of blocking_read be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_read",
+            "Histogram of the time spent on blocking_reading",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_read",
+            "blocking_read size",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_read =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_write
+        let opendal_requests_total_blocking_write = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_write",
+            "Total times of blocking_write be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_write",
+            "Histogram of the time spent on blocking_writing",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_requests_duration_seconds_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_bytes_total_blocking_write",
+            "total size by blocking_write",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+        let opendal_bytes_total_blocking_write =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_stat
+        let opendal_requests_total_blocking_stat = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_stat",
+            "Total times of blocking_stat be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_stat",
+            "blocking_stat letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_stat =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_delete
+        let opendal_requests_total_blocking_delete = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_delete",
+            "Total times of blocking_delete be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_delete",
+            "blocking_delete letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_delete =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_list
+        let opendal_requests_total_blocking_list = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_list",
+            "Total times of blocking_list be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_list",
+            "blocking_list letency",
+            exponential_buckets(0.01, 2.0, 16).unwrap()
+        );
+
+        let opendal_requests_duration_seconds_blocking_list =
+            register_histogram_vec_with_registry!(opts, &["schema"], registry).unwrap();
+
+        // blocking_scan
+        let opendal_requests_total_blocking_scan = register_int_counter_vec_with_registry!(
+            "opendal_requests_total_blocking_scan",
+            "Total times of blocking_scan be called",
+            &["schema"],
+            registry
+        )
+        .unwrap();
+
+        let opts = histogram_opts!(
+            "opendal_requests_duration_seconds_blocking_scan",

Review Comment:
   Notice that `metadata` is not included in `Operation`, so I just missed this one temporarily. You can let me add it as well if you want. 😄
   Is this OK? 
   ```
   # HELP requests_total_read Total times of read be called
   # TYPE requests_total_read counter
   requests_total_read{operation="read",scheme="memory"} 1
   # HELP requests_total_stat Total times of stat be called
   # TYPE requests_total_stat counter
   requests_total_stat{operation="stat",scheme="memory"} 1
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165721704


##########
core/src/raw/accessor.rs:
##########
@@ -398,6 +398,8 @@ impl Accessor for () {
             max_batch_operations: None,
             capabilities: None.into(),
             hints: None.into(),
+            #[cfg(feature = "layers-prometheus")]

Review Comment:
   I see.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1165797028


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,1240 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::GenericCounterVec;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_vec_with_registry;
+use prometheus::register_int_counter_vec_with_registry;
+use prometheus::Registry;
+use prometheus::{core::AtomicU64, HistogramVec};
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///     let registry = prometheus::default_registry();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer::with_registry(registry.clone()))
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Clone)]
+pub struct PrometheusLayer {
+    registry: Registry,
+}
+
+impl PrometheusLayer {
+    /// create PrometheusLayer by incoming registry.
+    pub fn with_registry(registry: Registry) -> Self {
+        Self { registry }
+    }
+}
+
+impl Default for PrometheusLayer {
+    fn default() -> Self {
+        Self {
+            registry: prometheus::Registry::default(),
+        }
+    }
+}
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let meta = inner.info();
+        let scheme = meta.scheme();
+
+        PrometheusAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(self.registry.clone())),
+            scheme: scheme.to_string(),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub requests_total_metadata: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_metadata: HistogramVec,
+
+    // create
+    pub requests_total_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_create: HistogramVec,
+
+    /// read
+    pub requests_total_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_read: HistogramVec,
+    pub bytes_total_read: HistogramVec,
+
+    // write
+    pub requests_total_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_write: HistogramVec,
+    pub bytes_total_write: HistogramVec,
+
+    // stat
+    pub requests_total_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_stat: HistogramVec,
+
+    // delete
+    pub requests_total_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_delete: HistogramVec,
+
+    // list
+    pub requests_total_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_list: HistogramVec,
+
+    // scan
+    pub requests_total_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_scan: HistogramVec,
+
+    // presign
+    pub requests_total_presign: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_presign: HistogramVec,
+
+    // batch
+    pub requests_total_batch: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_batch: HistogramVec,
+
+    // blocking create
+    pub requests_total_blocking_create: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_create: HistogramVec,
+
+    // blocking read
+    pub requests_total_blocking_read: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_read: HistogramVec,
+    pub bytes_total_blocking_read: HistogramVec,
+
+    // blocking write
+    pub requests_total_blocking_write: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_write: HistogramVec,
+    pub bytes_total_blocking_write: HistogramVec,
+
+    // blocking stat
+    pub requests_total_blocking_stat: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_stat: HistogramVec,
+
+    // blocking delete
+    pub requests_total_blocking_delete: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_delete: HistogramVec,
+
+    // blocking list
+    pub requests_total_blocking_list: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_list: HistogramVec,
+
+    // blocking scan
+    pub requests_total_blocking_scan: GenericCounterVec<AtomicU64>,
+    pub requests_duration_seconds_blocking_scan: HistogramVec,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata

Review Comment:
   Oh, `metadata` in `Accessor` just returns static info like `scheme`, `bucket_name` and so on. No real IO or syscall happened here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] wcy-fdu commented on pull request #1930: feat: add prometheus layer support

Posted by "wcy-fdu (via GitHub)" <gi...@apache.org>.
wcy-fdu commented on PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#issuecomment-1507916276

   > 
   
   Do you mean only remain three basic metrics(`requests_total`, `requests_duration_seconds`,`bytes_total`), and use label to distinguish different requests?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] ClSlaid commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "ClSlaid (via GitHub)" <gi...@apache.org>.
ClSlaid commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164281609


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,944 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use crate::ops::*;
+use crate::raw::*;
+use crate::*;
+
+use prometheus::{
+    core::{AtomicU64, GenericCounter},
+    register_int_counter_with_registry, Histogram,
+};
+use prometheus::{exponential_buckets, histogram_opts, register_histogram_with_registry, Registry};
+
+use crate::raw::Accessor;

Review Comment:
   Please group external crates with external crates, and in-crate imports with in-crate imports.
   
   Please do list items instead of grouping them in braces.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #1930: feat: add prometheus layer support

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1930:
URL: https://github.com/apache/incubator-opendal/pull/1930#discussion_r1164407594


##########
core/src/layers/prometheus.rs:
##########
@@ -0,0 +1,990 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fmt::Debug;
+use std::fmt::Formatter;
+use std::io;
+use std::sync::Arc;
+use std::task::Context;
+use std::task::Poll;
+
+use async_trait::async_trait;
+use bytes::Bytes;
+use futures::FutureExt;
+use futures::TryFutureExt;
+use log::debug;
+
+use prometheus::core::AtomicU64;
+use prometheus::core::GenericCounter;
+use prometheus::exponential_buckets;
+use prometheus::histogram_opts;
+use prometheus::register_histogram_with_registry;
+use prometheus::register_int_counter_with_registry;
+use prometheus::Histogram;
+use prometheus::Registry;
+
+use crate::ops::*;
+use crate::raw::Accessor;
+use crate::raw::*;
+use crate::*;
+/// Add [prometheus](https://docs.rs/prometheus) for every operations.
+///
+/// # Examples
+///
+/// ```
+/// use log::debug;
+/// use log::info;
+/// use opendal::services;
+/// use opendal::Operator;
+/// use opendal::Result;
+///
+/// use opendal::layers::PrometheusLayer;
+/// use prometheus::Encoder;
+///
+/// /// Visit [`opendal::services`] for more service related config.
+/// /// Visit [`opendal::Object`] for more object level APIs.
+/// #[tokio::main]
+/// async fn main() -> Result<()> {
+///     // Pick a builder and configure it.
+///     let builder = services::Memory::default();
+///
+///     let op = Operator::new(builder)
+///         .expect("must init")
+///         .layer(PrometheusLayer)
+///         .finish();
+///     debug!("operator: {op:?}");
+///
+///     // Write data into object test.
+///     op.write("test", "Hello, World!").await?;
+///     // Read data from object.
+///     let bs = op.read("test").await?;
+///     info!("content: {}", String::from_utf8_lossy(&bs));
+///
+///     // Get object metadata.
+///     let meta = op.stat("test").await?;
+///     info!("meta: {:?}", meta);
+///
+///     // Export prometheus metrics.
+///     let mut buffer = Vec::<u8>::new();
+///     let encoder = prometheus::TextEncoder::new();
+///     encoder.encode(&prometheus::gather(), &mut buffer).unwrap();
+///     println!("## Prometheus Metrics");
+///     println!("{}", String::from_utf8(buffer.clone()).unwrap());
+///     Ok(())
+/// }
+/// ```
+#[derive(Debug, Copy, Clone)]
+pub struct PrometheusLayer;
+
+impl<A: Accessor> Layer<A> for PrometheusLayer {
+    type LayeredAccessor = PrometheusMetricsAccessor<A>;
+
+    fn layer(&self, inner: A) -> Self::LayeredAccessor {
+        let registry = prometheus::default_registry();
+
+        PrometheusMetricsAccessor {
+            inner,
+            stats: Arc::new(PrometheusMetrics::new(registry.clone())),
+        }
+    }
+}
+/// [`PrometheusMetrics`] provide the performance and IO metrics.
+#[derive(Debug)]
+pub struct PrometheusMetrics {
+    // metadata
+    pub metadata_request_counts: GenericCounter<AtomicU64>,
+    pub metadata_request_latency: Histogram,
+
+    // create
+    pub create_request_counts: GenericCounter<AtomicU64>,
+    pub create_request_latency: Histogram,
+
+    /// read
+    pub read_request_counts: GenericCounter<AtomicU64>,
+    pub read_request_latency: Histogram,
+    pub read_size: Histogram,
+
+    // write
+    pub write_request_counts: GenericCounter<AtomicU64>,
+    pub write_request_latency: Histogram,
+    pub write_size: Histogram,
+
+    // stat
+    pub stat_request_counts: GenericCounter<AtomicU64>,
+    pub stat_request_latency: Histogram,
+
+    // delete
+    pub delete_request_counts: GenericCounter<AtomicU64>,
+    pub delete_request_latency: Histogram,
+
+    // list
+    pub list_request_counts: GenericCounter<AtomicU64>,
+    pub list_request_latency: Histogram,
+
+    // scan
+    pub scan_request_counts: GenericCounter<AtomicU64>,
+    pub scan_request_latency: Histogram,
+
+    // presign
+    pub presign_request_counts: GenericCounter<AtomicU64>,
+    pub presign_request_latency: Histogram,
+
+    // batch
+    pub batch_request_counts: GenericCounter<AtomicU64>,
+    pub batch_request_latency: Histogram,
+
+    // blocking create
+    pub blocking_create_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_create_request_latency: Histogram,
+
+    // blocking read
+    pub blocking_read_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_read_request_latency: Histogram,
+    pub blocking_read_size: Histogram,
+
+    // blocking write
+    pub blocking_write_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_write_request_latency: Histogram,
+    pub blocking_write_size: Histogram,
+
+    // blocking stat
+    pub blocking_stat_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_stat_request_latency: Histogram,
+
+    // blocking delete
+    pub blocking_delete_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_delete_request_latency: Histogram,
+
+    // blocking list
+    pub blocking_list_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_list_request_latency: Histogram,
+
+    // blocking scan
+    pub blocking_scan_request_counts: GenericCounter<AtomicU64>,
+    pub blocking_scan_request_latency: Histogram,
+}
+
+impl PrometheusMetrics {
+    /// new with prometheus register.
+    pub fn new(registry: Registry) -> Self {
+        // metadata
+        let metadata_request_counts = register_int_counter_with_registry!(
+            "metadata_request_counts",

Review Comment:
   How about keeping the similiar naming style like `opendal_requests_total`?
   
   And does prometheus have the silimar concept of `label`? How about keeping our metrics organized in different label instead?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org