[jira] [Updated] (KUDU-2995) Tuning kudu write performance

["wxmimperio (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

wxmimperio updated KUDU-2995:
-----------------------------
    Issue Type: Test  (was: Wish)

> Tuning kudu write performance
> -----------------------------
>
>                 Key: KUDU-2995
>                 URL: https://issues.apache.org/jira/browse/KUDU-2995
>             Project: Kudu
>          Issue Type: Test
>          Components: client
>    Affects Versions: 1.7.0
>         Environment: kudu 1.7.0-cdh5.16.2
>            Reporter: wxmimperio
>            Priority: Major
>
> Kudu version： kudu 1.7.0-cdh5.16.2
> System memary pre node：256G
> 4 SSDs per machine：512G
> Three Master nodes and three Tserver nodes.
> {code:java}
> // Master config
> --fs_wal_dir=/mnt/disk1/kudu/var/wal
> --fs_data_dirs=/mnt/disk1/kudu/var/data,/mnt/disk2/kudu/var/data,/mnt/disk3/kudu/var/data,/mnt/disk4/kudu/var/data
> --fs_metadata_dir=/mnt/disk1/kudu/var/metadata
> --log_dir=/mnt/disk1/kudu/var/logs
> --master_addresses=xxxx
> --maintenance_manager_num_threads=2
> --block_cache_capacity_mb=6144
> --memory_limit_hard_bytes=34359738368
> --max_log_size=40
> // Tserver config
> --fs_wal_dir=/mnt/disk1/kudu/var/wal
> --fs_data_dirs=/mnt/disk1/kudu/var/data,/mnt/disk2/kudu/var/data,/mnt/disk3/kudu/var/data,/mnt/disk4/kudu/var/data
> --fs_metadata_dir=/mnt/disk1/kudu/var/metadata
> --log_dir=/mnt/disk1/kudu/var/logs
> --tserver_master_addrs=xxxx
> --block_cache_capacity_mb=6144
> --memory_limit_hard_bytes=34359738368
> --max_log_size=40
> // Table schema
> // _key is UUID for each msg
> // event_time is data time
> // Schema has only 15 columns
> // Single message does not exceed 100Bytes
> HASH (_key) PARTITIONS 3,
> RANGE (event_time) (
>     PARTITION 2019-10-31T16:00:00.000000Z <= VALUES < 2019-11-30T16:00:00.000000Z
> )
> {code}
> I write a project to write data to kudu.
> Whether manual or automatic flush mode write speed is only 6M/s.
> I think SSD should be more than this speed, and the network and memory have not reached the bottleneck.
> Is this the normal level of kudu writing? How to tuning?
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)