[jira] [Resolved] (IMPALA-8781) Add additional tests in test_result_spooling.py and validate cancellation logic

["Sahil Takiar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sahil Takiar resolved IMPALA-8781.
----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 3.3.0

Commit Hash: bbec8fa74961755269298706302477780019e7d5

IMPALA-8781: Result spooling tests to cover edge cases and cancellation

Adds additional tests to test_result_spooling.py to cover various edge
cases when fetching query results (ensure all Impala types are returned
properly, UDFs are evaluated correctly, etc.). A new QueryTest file
result-spooling.test is added to encapsulate all these tests. Tests with
a decreased ROW_BATCH_SIZE are added as well to validate that
BufferedPlanRootSink buffers row batches correctly.

BufferedPlanRootSink requires careful synchronization of the producer
and consumer threads, especially when queries are cancelled. The
TestResultSpoolingCancellation class is dedicated to running
cancellation tests with SPOOL_QUERY_RESULTS = true. The implementation
is heavily borrowed from test_cancellation.py and some of the logic is
re-factored into a new utility class called cancel_utils.py to avoid
code duplication between test_cancellation.py and
test_result_spooling.py.

Testing:
* Looped test_result_spooling.py overnight with no failures
* Core tests passed

Change-Id: Ib3b3a1539c4a5fa9b43c8ca315cea16c9701e283
Reviewed-on: http://gerrit.cloudera.org:8080/13907
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>

> Add additional tests in test_result_spooling.py and validate cancellation logic
> -------------------------------------------------------------------------------
>
>                 Key: IMPALA-8781
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8781
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>            Priority: Major
>             Fix For: Impala 3.3.0
>
>
> {{test_result_spooling.py}} currently runs a few basic tests with result spooling enabled. We should add some more to cover all necessary edge cases (ensure all Impala types are returned correctly, UDFs are evaluated correctly, etc.) and add tests to validate the cancellation logic in {{PlanRootSink}}.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org