[GitHub] [hadoop-ozone] elek opened a new pull request #59: HDDS-2327. Provide new Freon test to test Ratis pipeline with pure XceiverClientRatis

[GitBox <gi...@apache.org>]

elek opened a new pull request #59: HDDS-2327. Provide new Freon test to test Ratis pipeline with pure XceiverClientRatis
URL: https://github.com/apache/hadoop-ozone/pull/59
 
 
   ## What changes were proposed in this pull request?
   
   
   
   Xiaoyu Yao suggested during an offline talk to implement one additional Freon test to test the ratis part only.
   
   It can use XceiverClientManager which creates a pure XceiverClientRatis. The client can be used to generate chunks as the datanode accepts any container id / block id.
   
   With this approach we can stress-test one selected ratis pipeline without having full end2end overhead of the key creation (OM, SCM, etc.)
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-2327
   
   ## How was this patch tested?
   
   ```
   cd hadoop-ozone/dist/target/ozone-*/compose/ozone
   docker-compose up -d --scale datanode=3
   #wait
   docker-compose exec scm ozone freon dcg -n1000
   ```
   
   Execution should be fine. Eg:
   ```
   10/18/19, 12:50:28 PM ==========================================================
   
   -- Timers ----------------------------------------------------------------------
   file-create
                count = 1000
            mean rate = 142.16 calls/second
        1-minute rate = 140.00 calls/second
        5-minute rate = 140.00 calls/second
       15-minute rate = 140.00 calls/second
                  min = 0.09 milliseconds
                  max = 689.10 milliseconds
                 mean = 63.18 milliseconds
               stddev = 63.59 milliseconds
               median = 58.54 milliseconds
                 75% <= 77.92 milliseconds
                 95% <= 110.45 milliseconds
                 98% <= 133.97 milliseconds
                 99% <= 175.51 milliseconds
               99.9% <= 675.39 milliseconds
   
   
   Total execution time (sec): 7
   Failures: 0
   Successful executions: 1000
   ```
   
   Check the number of chunk files on the system:
   
   ```
    for i in `seq 1 3`; do docker exec ozone_datanode_1 find  /data/ -type f; done | grep chunk | wc
   3000
   ``` 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org