[jira] [Updated] (HBASE-27088) IntegrationLoadTestCommonCrawl async load improvements

["Andrew Kyle Purtell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-27088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Kyle Purtell updated HBASE-27088:
----------------------------------------
    Hadoop Flags: Reviewed
      Resolution: Fixed
          Status: Resolved  (was: Patch Available)

> IntegrationLoadTestCommonCrawl async load improvements
> ------------------------------------------------------
>
>                 Key: HBASE-27088
>                 URL: https://issues.apache.org/jira/browse/HBASE-27088
>             Project: HBase
>          Issue Type: Improvement
>          Components: integration tests, test
>            Reporter: Andrew Kyle Purtell
>            Assignee: Andrew Kyle Purtell
>            Priority: Minor
>             Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> ITLCC improvements:
> - Use an async client and work stealing executor for parallelism during loads.
> - Remove the verification read retries, these are not that effective during replication lag anyway.
> - Increase max task attempts because S3 might throttle.
> - Implement a side task that exercises Increments by extracting urls from content and updating a cf that tracks referrer counts. These are not validated at this time. It could be possible to log the increments, sum them with a reducer, and then verify the total, but this is left as a future exercise.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)