Re: [I] BUG: metrics "opendal_operation_duration_seconds_count" gets HTTP request count for chunk reads [opendal]

["dentiny (via GitHub)" <gi...@apache.org>]

dentiny commented on issue #7339:
URL: https://github.com/apache/opendal/issues/7339#issuecomment-4166161960

   OK I found the read and write path are handled completely differently.
   
   Write path
   ```sh
   Operator::write("path", data)
     → WriteContext::create()
       → ctx.acc.write(path, args)          ← called ONCE, goes through all layers
         → MetricsAccessor::write()         ← creates ONE MetricsWrapper<Writer>
           → inner.write()
             → S3Backend::write()           ← returns MultipartWriter<S3Writer>
       → returns oio::Writer (wrapped by MetricsWrapper)
     → WriteGenerator holds this single writer
     → WriteGenerator::write(chunk1)        ← calls w.write(chunk1) on the SAME writer
       → MetricsWrapper::write()            ← accumulates bytes
         → MultipartWriter::write()
           → S3Writer::write_part()         ← HTTP PUT (internal, below layers)
     → WriteGenerator::write(chunk2)        ← same writer, same MetricsWrapper
       → MetricsWrapper::write()            ← accumulates more bytes
         → MultipartWriter::write()
           → S3Writer::write_part()         ← another HTTP PUT (internal)
     → WriteGenerator::close()
       → MetricsWrapper::close()
         → MultipartWriter::close()
           → S3Writer::complete_part()      ← HTTP POST (internal)
     → MetricsWrapper drops → emits ONE aggregated operation metric
   ```
   
   Read path
   ```sh
   Operator::reader("path").read(..)
     → Reader::new(ReadContext { acc, path, args, options })
     → BufferStream → ReadGenerator::next_reader()
       → ctx.acc.read(path, args_for_chunk_1)    ← through ALL layers
         → MetricsAccessor::read()               ← creates MetricsWrapper #1
           → S3Backend::read() → HTTP GET
     → ReadGenerator::next_reader()
       → ctx.acc.read(path, args_for_chunk_2)    ← through ALL layers AGAIN
         → MetricsAccessor::read()               ← creates MetricsWrapper #2
           → S3Backend::read() → HTTP GET
     → MetricsWrapper #1 drops → emits metric for chunk 1
     → MetricsWrapper #2 drops → emits metric for chunk 2
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org