[GitHub] [incubator-doris] morningman commented on a change in pull request #7134: [enhancement] display load bytes in show load stmt

[GitBox <gi...@apache.org>]

morningman commented on a change in pull request #7134:
URL: https://github.com/apache/incubator-doris/pull/7134#discussion_r752399627



##########
File path: fe/fe-core/src/main/java/org/apache/doris/load/loadv2/BrokerLoadJob.java
##########
@@ -346,6 +346,16 @@ private void updateLoadingStatus(BrokerLoadingTaskAttachment attachment) {
         }
     }
 
+    @Override
+    public void updateProgress(Long beId, TUniqueId loadId, TUniqueId fragmentId, long scannedRows,
+                               long scannedBytes, boolean isDone) {
+        super.updateProgress(beId, loadId, fragmentId, scannedRows, scannedBytes, isDone);
+        progress = (int) ((double) loadStatistic.getLoadBytes() / loadStatistic.totalFileSizeB * 100);

Review comment:
       This may always larger than 100, when loading parquet or orc format files.

##########
File path: be/src/exec/tablet_sink.cpp
##########
@@ -707,13 +707,15 @@ Status OlapTableSink::open(RuntimeState* state) {
 
 Status OlapTableSink::send(RuntimeState* state, RowBatch* input_batch) {
     SCOPED_TIMER(_profile->total_time_counter());
-    _number_input_rows += input_batch->num_rows();
     // update incrementally so that FE can get the progress.
     // the real 'num_rows_load_total' will be set when sink being closed.
-    state->update_num_rows_load_total(input_batch->num_rows());
-    state->update_num_bytes_load_total(input_batch->total_byte_size());
-    DorisMetrics::instance()->load_rows->increment(input_batch->num_rows());
-    DorisMetrics::instance()->load_bytes->increment(input_batch->total_byte_size());
+    int64_t num_rows = input_batch->num_rows();
+    int64_t num_bytes = input_batch->total_byte_size();

Review comment:
       Pay attention that `total_byte_size()` may cost times.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org