You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2024/01/10 14:16:00 UTC

[jira] [Updated] (HIVE-27693) FileSystem counters summary to include durations

     [ https://issues.apache.org/jira/browse/HIVE-27693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

László Bodor updated HIVE-27693:
--------------------------------
    Description: 
given this:
{code}
INFO  : FileSystem Counters Summary
INFO  : 
INFO  : Scheme: HDFS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1        350.69MB           622                  0                 0B             0
INFO  :     Map 10              0B             0                  0                 0B             0
INFO  :     Map 11              0B             1                  0                 0B             0
INFO  :     Map 12              0B             1                  0                 0B             0
INFO  :     Map 13              0B             1                  0                 0B             0
INFO  :     Map 14            206B             1                  0                 0B             0
INFO  :     Map 15            206B             1                  0                 0B             0
INFO  :     Map 16              0B             1                  0                 0B             0
INFO  :     Map 17              0B             1                  0                 0B             0
{code}

I want to be able to see how much time has actually been spent on filesystem operation on a specific scheme, as by looking at that, I could easily tell if file reads were slow

  was:
given this:
{code}
INFO  : FileSystem Counters Summary
INFO  : 
INFO  : Scheme: HDFS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1        350.69MB           622                  0                 0B             0
INFO  :     Map 10              0B             0                  0                 0B             0
INFO  :     Map 11              0B             1                  0                 0B             0
INFO  :     Map 12              0B             1                  0                 0B             0
INFO  :     Map 13              0B             1                  0                 0B             0
INFO  :     Map 14            206B             1                  0                 0B             0
INFO  :     Map 15            206B             1                  0                 0B             0
INFO  :     Map 16              0B             1                  0                 0B             0
INFO  :     Map 17              0B             1                  0                 0B             0
{code}

I want to be able to see how much time has actually been spent on filesystem operation on a specific scheme, as b looking at that, I could easily tell if file reads were slow


> FileSystem counters summary to include durations 
> -------------------------------------------------
>
>                 Key: HIVE-27693
>                 URL: https://issues.apache.org/jira/browse/HIVE-27693
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: László Bodor
>            Priority: Major
>
> given this:
> {code}
> INFO  : FileSystem Counters Summary
> INFO  : 
> INFO  : Scheme: HDFS
> INFO  : ----------------------------------------------------------------------------------------------
> INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
> INFO  : ----------------------------------------------------------------------------------------------
> INFO  :      Map 1        350.69MB           622                  0                 0B             0
> INFO  :     Map 10              0B             0                  0                 0B             0
> INFO  :     Map 11              0B             1                  0                 0B             0
> INFO  :     Map 12              0B             1                  0                 0B             0
> INFO  :     Map 13              0B             1                  0                 0B             0
> INFO  :     Map 14            206B             1                  0                 0B             0
> INFO  :     Map 15            206B             1                  0                 0B             0
> INFO  :     Map 16              0B             1                  0                 0B             0
> INFO  :     Map 17              0B             1                  0                 0B             0
> {code}
> I want to be able to see how much time has actually been spent on filesystem operation on a specific scheme, as by looking at that, I could easily tell if file reads were slow



--
This message was sent by Atlassian Jira
(v8.20.10#820010)