[jira] [Resolved] (IMPALA-4703) Add reservation stress option for test coverage

["Tim Armstrong (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-4703.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.10.0



IMPALA-4703: reservation denial debug action

Add debug action to deny reservation increases with some probability.
This allows us to test various scenarios, particularly:
* The case when the node only gets its initial reservation and must
  run to completion without increasing its reservation.
* The case when there is some memory pressure and the node sometimes
  gets a reservation increase and sometimes doesn't.

E.g. to deny all reservation requests after an ExecNode has opened:

  set debug_action=-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0

This was applied to test_spilling. It caught a bug in the PAGG
with spilling string aggregations.

This required some minor extensions to the debug actions.
* Allow debug actions that apply to all ExecNodes if node_id is -1.
* Allow passing parameters to debug actions. The current grammar of the
  actions is not well-oriented towards extension, so I resorted to using
  @ as a new delimiter.

I also optimised ExecDebugAction() so that it is much faster in the
common case and extended --disable_mem_pools to prevent the buffer pool
from holding onto unused buffers.

Change-Id: Ied39bb091b12156e5dc61b528c6c0cd8de3fe657
Reviewed-on: http://gerrit.cloudera.org:8080/7022
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins

> Add reservation stress option for test coverage
> -----------------------------------------------
>
>                 Key: IMPALA-4703
>                 URL: https://issues.apache.org/jira/browse/IMPALA-4703
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>    Affects Versions: Impala 2.8.0
>            Reporter: Dan Hecht
>            Assignee: Tim Armstrong
>            Priority: Minor
>             Fix For: Impala 2.10.0
>
>
> We should add a stress option to allow us to exercise various reservation scenarios. e.g.:
> # Initial reservation not granted -> reservation error status in Prepare().
> # All reservation requests beyond initial not granted -> all (valid) queries complete successfully.
> # Random % of reservation requests beyond initial not granted -> all (valid) queries complete successfully.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)