You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ma...@apache.org on 2016/07/01 16:54:16 UTC
incubator-airflow git commit: [AIRFLOW-284] HiveServer2Hook fix for
cursor scope for get_results
Repository: incubator-airflow
Updated Branches:
refs/heads/master 0a94cae4b -> a9f9fe560
[AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results
Dear Airflow Maintainers,
Please accept this PR that addresses the following issues:
- *(replace with a link to AIRFLOW-X)*
Detail:
The function `get_results` implemented in the `HiveServer2Hook` does not execute multiple commands passed to it in a list, in the singular cursor scope. This has caused SQL statements that depend on the execution of `add jar` and `set` commands to fail as they are being executed in different cursor scopes which are not persistent.
The code has been updated to have the cursor object persistent.
Testing Done:
Reminders for contributors:
* Your PR's title must reference an issue on [Airflow's JIRA](https://issues.apache.org/jira/browse/AIRFLOW/). For example, a PR called "[AIRFLOW-1] My Amazing PR" would close JIRA issue #1. Please open a new issue if required!
* Please squash your commits when possible and follow the [7 rules of good Git commits](http://chris.beams.io/posts/git-commit/#seven-rules).
mistercrunch
Closes #1629 from sherwian/hivehook_fx
Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/a9f9fe56
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/a9f9fe56
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/a9f9fe56
Branch: refs/heads/master
Commit: a9f9fe560a7ca7bf03ee577616ab38344e0d73b7
Parents: 0a94cae
Author: sherwian_williamson <sh...@airbnb.com>
Authored: Fri Jul 1 09:52:55 2016 -0700
Committer: Maxime Beauchemin <ma...@gmail.com>
Committed: Fri Jul 1 09:53:19 2016 -0700
----------------------------------------------------------------------
airflow/hooks/hive_hooks.py | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/a9f9fe56/airflow/hooks/hive_hooks.py
----------------------------------------------------------------------
diff --git a/airflow/hooks/hive_hooks.py b/airflow/hooks/hive_hooks.py
index e8e10b4..eaad390 100644
--- a/airflow/hooks/hive_hooks.py
+++ b/airflow/hooks/hive_hooks.py
@@ -527,22 +527,22 @@ class HiveServer2Hook(BaseHook):
'data': [],
'header': [],
}
+ cur = conn.cursor()
for statement in hql:
- with conn.cursor() as cur:
- cur.execute(statement)
- records = []
- try:
- # impala Lib raises when no results are returned
- # we're silencing here as some statements in the list
- # may be `SET` or DDL
- records = cur.fetchall()
- except ProgrammingError:
- logging.debug("get_results returned no records")
- if records:
- results = {
- 'data': records,
- 'header': cur.description,
- }
+ cur.execute(statement)
+ records = []
+ try:
+ # impala Lib raises when no results are returned
+ # we're silencing here as some statements in the list
+ # may be `SET` or DDL
+ records = cur.fetchall()
+ except ProgrammingError:
+ logging.debug("get_results returned no records")
+ if records:
+ results = {
+ 'data': records,
+ 'header': cur.description,
+ }
return results
def to_csv(