[jira] [Created] (IMPALA-8452) Avro scanner seems broken

["Michael Ho (JIRA)" <ji...@apache.org>]

Michael Ho created IMPALA-8452:
----------------------------------

             Summary: Avro scanner seems broken
                 Key: IMPALA-8452
                 URL: https://issues.apache.org/jira/browse/IMPALA-8452
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 3.3.0
            Reporter: Michael Ho
            Assignee: Attila Jeges


A few scanner tests started failing recently on Centos6. Coincidentally, both of them only started happening after [this commit|https://github.com/apache/impala/commit/b5805de3e65fd1c7154e4169b323bb38ddc54f4f]. [~attilaj], can you please take a look and reassign if you think that commit is unrelated ? 

Oddly enough, this has shown up on Centos6. Other exhaustive runs with Centos7 seem to work fine. May be it's related to some platform's library ?

In the first case, a select count star from an avro table hangs for 2 hours:
{noformat}
query_test/test_scanners_fuzz.py:83: in test_fuzz_alltypes
    self.run_fuzz_test(vector, src_db, table_name, unique_database, table_name)
query_test/test_scanners_fuzz.py:201: in run_fuzz_test
    result = self.execute_query(query, query_options = query_options)
common/impala_test_suite.py:619: in wrapper
    return function(*args, **kwargs)
common/impala_test_suite.py:650: in execute_query
    return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:721: in __execute_query
    return impalad_client.execute(query, user=user)
common/impala_connection.py:180: in execute
    return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:183: in execute
    handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:360: in __execute_query
    self.wait_for_finished(handle)
beeswax/impala_beeswax.py:384: in wait_for_finished
    time.sleep(0.05)
E   Failed: Timeout >7200s

SET client_identifier=query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[protocol:beeswax|exec_option:{'debug_action':None;'abort_on_error':False;'mem_limit':'512m';'num_nodes':0}|table_format:avro/none];
SET batch_size=1;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=True;
SET abort_on_error=False;
SET mem_limit=512m;
-- executing against localhost:21000
select count(*) from test_fuzz_alltypes_2cdcb963.alltypes q;

-- 2019-04-24 04:14:31,857 INFO     MainThread: Started query 2049069f9f5e3aa8:f2fd47ff00000000
{noformat}

The second case has to do with incorrect number of rows in a select count star from tpch_avro.lineitem:
{noformat}
query_test/test_scanners.py:947: in test_tpch_scan_ranges
    self.run_test_case('tpch-scan-range-lengths', vector)
common/impala_test_suite.py:517: in run_test_case
    self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:370: in __verify_results_and_errors
    replace_filenames_with_placeholder)
common/test_result_verifier.py:449: in verify_raw_results
    VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:271: in verify_query_result_is_equal
    assert expected_results == actual_results
E   assert Comparing QueryTestResults (expected vs actual):
E     6001215 != 6000679

-- 2019-04-24 03:43:42,805 INFO     MainThread: max_scan_range_length=8412307
SET client_identifier=query_test/test_scanners.py::TestTpchScanRangeLengths::()::test_tpch_scan_ranges[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|;
-- executing against localhost:21000
use tpch_avro;

-- 2019-04-24 03:43:42,814 INFO     MainThread: Started query c04e1968443b52fc:5b99b1b300000000
SET client_identifier=query_test/test_scanners.py::TestTpchScanRangeLengths::()::test_tpch_scan_ranges[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET max_scan_range_length=8412307;
SET exec_single_node_rows_threshold=0;
-- executing against localhost:21000
select count(*)
from lineitem;
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org