You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/09/05 20:13:00 UTC
[jira] [Commented] (IMPALA-7490) Uninitialized variable in
data-load.py causes misleading error messages
[ https://issues.apache.org/jira/browse/IMPALA-7490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604882#comment-16604882 ]
ASF subversion and git services commented on IMPALA-7490:
---------------------------------------------------------
Commit 357c0a959d1a3a9cb2662773a407acfee3ca6fe2 in impala's branch refs/heads/master from stiga-huang
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=357c0a9 ]
IMPALA-7490: fix uninitialized variables in load-data.py
Fixes use of an uninitialized variable in bin/load-data.py
I found the following error message in a failed build, which is quite
misleading:
Traceback (most recent call last):
File "bin/load-data.py", line 495, in <module>
if __name__ == "__main__": main()
File "bin/load-data.py", line 459, in main
impala_exec_query_files_parallel(thread_pool, impala_create_files)
File "bin/load-data.py", line 297, in impala_exec_query_files_parallel
exec_query_files_parallel(thread_pool, query_files, 'impala')
File "bin/load-data.py", line 291, in exec_query_files_parallel
for result in thread_pool.imap_unordered(execution_function, query_files):
File "/usr/lib/python2.7/multiprocessing/pool.py", line 659, in next
raise value
UnboundLocalError: local variable 'query' referenced before assignment
The error is trown from the 'execution_function' which is
'exec_impala_query_from_file' in my case. Should not use 'query' if it's
undefined.
Change-Id: If0dd56a9b78a60b3a9f04d9f61e93b4b5d066b76
Reviewed-on: http://gerrit.cloudera.org:8080/11330
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Uninitialized variable in data-load.py causes misleading error messages
> -----------------------------------------------------------------------
>
> Key: IMPALA-7490
> URL: https://issues.apache.org/jira/browse/IMPALA-7490
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> I found the following messages in a failed data-loading.
> {code}
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-hbase-none-none.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-rc-snap-block.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-text-none-none.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-avro-snap-block.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-kudu-none-none.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-orc-def-block.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-seq-snap-block.sql
> 19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-parquet-none-none.sql
> Traceback (most recent call last):
> File "bin/load-data.py", line 495, in <module>
> if __name__ == "__main__": main()
> File "bin/load-data.py", line 459, in main
> impala_exec_query_files_parallel(thread_pool, impala_create_files)
> File "bin/load-data.py", line 297, in impala_exec_query_files_parallel
> exec_query_files_parallel(thread_pool, query_files, 'impala')
> File "bin/load-data.py", line 291, in exec_query_files_parallel
> for result in thread_pool.imap_unordered(execution_function, query_files):
> File "/usr/lib/python2.7/multiprocessing/pool.py", line 659, in next
> raise value
> UnboundLocalError: local variable 'query' referenced before assignment
> {code}
> The error is misleading and actually don't tell anything (e.g. log file locations) for debug. The real cause for my error is that my Impala cluster failed to launch thus causes bin/load-data.py to fail.
> The 'query' variable is used in the Exception clause inside 'exec_impala_query_from_file'. We should judge whether it's initialized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org