You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tianyi Wang (JIRA)" <ji...@apache.org> on 2017/11/13 18:54:00 UTC
[jira] [Resolved] (IMPALA-6164) test_always_false_filter failure on
ASAN
[ https://issues.apache.org/jira/browse/IMPALA-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tianyi Wang resolved IMPALA-6164.
---------------------------------
Resolution: Fixed
IMPALA-6164: Fix stale query profile in TestAlwaysFalseFilter
TestAlwaysFalseFilter gets the query profile without fetching all the
rows, resulting in a stale query profile and failing the test. With
this patch all the rows are fetched before getting the query profile.
This is enough to get the final profile because the query profile
finalization is performed in Coordinator::GetNext after we hit eos.
A bug in Base64Decode related to query profile decoding is also fixed.
Currently Base64Decode may produce incorrect output length if the output
parameter is not initialized with 0.
Testing: TestAlwaysFalseFilter is run and passes 1000 times. It doesn't
pass 1000 times consecutively without this patch.
Change-Id: I04bb76d20541fa035d88167b593d1b8bc3873e89
Reviewed-on: http://gerrit.cloudera.org:8080/8498
Reviewed-by: Alex Behm <al...@cloudera.com>
Tested-by: Impala Public Jenkins
> test_always_false_filter failure on ASAN
> ----------------------------------------
>
> Key: IMPALA-6164
> URL: https://issues.apache.org/jira/browse/IMPALA-6164
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.11.0
> Reporter: Thomas Tauber-Marshall
> Assignee: Tianyi Wang
> Priority: Blocker
> Labels: broken-build
>
> Seen in an asf-master ASAN job at git hash 6b37a793a0af74ba01c67bb5ac7285af7c5c4d52:
> {noformat}
> 04:15:21 ____________________ TestAlwaysFalseFilter.test_skip_split _____________________
> 04:15:21
> 04:15:21 self = <test_always_false_filter.TestAlwaysFalseFilter object at 0x347a810>
> 04:15:21 cursor = <impala.hiveserver2.HiveServer2Cursor object at 0x3246310>
> 04:15:21
> 04:15:21 @SkipIfBuildType.not_dev_build
> 04:15:21 @pytest.mark.execute_serially
> 04:15:21 @CustomClusterTestSuite.with_args("--skip_file_runtime_filtering=true")
> 04:15:21 def test_skip_split(self, cursor):
> 04:15:21 """IMPALA-5789: Test that always false filter filters out splits when file-level
> 04:15:21 filtering is disabled. The filtering is not enabled in seq-based file formats."""
> 04:15:21 cursor.execute("SET RUNTIME_FILTER_MODE=GLOBAL")
> 04:15:21 cursor.execute("SET RUNTIME_FILTER_WAIT_TIME_MS=10000")
> 04:15:21 query = """select STRAIGHT_JOIN * from alltypes inner join
> 04:15:21 (select * from alltypessmall where smallint_col=-1) v
> 04:15:21 on v.year = alltypes.year"""
> 04:15:21 # Manually iterate through file formats instead of creating a test matrix to prevent
> 04:15:21 # the cluster from restarting multiple times.
> 04:15:21 for table_suffix in ['_avro', '_rc', '_seq']:
> 04:15:21 cursor.execute("use functional" + table_suffix)
> 04:15:21 cursor.execute(query)
> 04:15:21 profile = cursor.get_profile()
> 04:15:21 assert re.search("Files rejected: [^0] \([^0]\)", profile) is None
> 04:15:21 assert re.search("Splits rejected: [^0] \([^0]\)", profile) is None
> 04:15:21 for table_suffix in ['', '_parquet']:
> 04:15:21 cursor.execute("use functional" + table_suffix)
> 04:15:21 cursor.execute(query)
> 04:15:21 profile = cursor.get_profile()
> 04:15:21 assert re.search("Files rejected: [^0] \([^0]\)", profile) is None
> 04:15:21 > assert re.search("Splits rejected: 8 \(8\)", profile) is not None
> 04:15:21 E assert None is not None
> 04:15:21 E + where None = <function search at 0x1076cf8>('Splits rejected: 8 \\(8\\)', 'Query (id=b1414c2cf5045bec:4e93cc3200000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil...ryContextSwitches: 7 (7)\n - TotalRawHdfsReadTime(*): 4.999ms\n - TotalReadThroughput: 0.00 /sec\n')
> 04:15:21 E + where <function search at 0x1076cf8> = re.search
> 04:15:21
> 04:15:21 custom_cluster/test_always_false_filter.py:53: AssertionError
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)