You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "David Morel (JIRA)" <ji...@apache.org> on 2017/03/11 00:52:04 UTC
[jira] [Resolved] (AIRFLOW-892) HIvePartitionSensor and
HiveNamedPartitionSensor raise AttributeError
[ https://issues.apache.org/jira/browse/AIRFLOW-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Morel resolved AIRFLOW-892.
---------------------------------
Resolution: Fixed
Fix Version/s: Airflow 1.8
> HIvePartitionSensor and HiveNamedPartitionSensor raise AttributeError
> ---------------------------------------------------------------------
>
> Key: AIRFLOW-892
> URL: https://issues.apache.org/jira/browse/AIRFLOW-892
> Project: Apache Airflow
> Issue Type: Bug
> Components: core
> Affects Versions: Airflow 1.8
> Environment: Virtualenv 2.7.13 on Centos 6 / Centos 6 + Docker
> Reporter: David Morel
> Priority: Critical
> Labels: metastore
> Fix For: Airflow 1.8
>
> Attachments: airflow-hivesensor.diff
>
>
> All HiveParititionSensor and friends using the Meatastore hook throw this exception in 2 installs I've done, one on a clean docker to rule out other causes. install has been done using pip install git+https://github.com/apache/incubator-airflow.git@v1-8-stable
> {noformat}
> [2017-02-22 08:44:49,079] {base_task_runner.py:95} INFO - Subtask: --------------------------------------------------------------------------------
> [2017-02-22 08:44:49,079] {base_task_runner.py:95} INFO - Subtask: Starting attempt 1 of 1
> [2017-02-22 08:44:49,079] {base_task_runner.py:95} INFO - Subtask: --------------------------------------------------------------------------------
> [2017-02-22 08:44:49,079] {base_task_runner.py:95} INFO - Subtask:
> [2017-02-22 08:44:49,111] {base_task_runner.py:95} INFO - Subtask: [2017-02-22 08:44:49,110] {models.py:1342} INFO - Executing <Task(HivePartitionSensor): wait_for_***> on 2017-02-18 00:00:00
> [2017-02-22 08:44:49,124] {base_task_runner.py:95} INFO - Subtask: [2017-02-22 08:44:49,123] {sensors.py:373} INFO - Poking for table ****, partition d_date='2017-02-18'
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: [2017-02-22 08:44:49,124] {models.py:1417} ERROR -
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent call last):
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: result = task_copy.execute(context=context)
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/operators/sensors.py", line 76, in execute
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: while not self.poke(context):
> [2017-02-22 08:44:49,125] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/operators/sensors.py", line 375, in poke
> [2017-02-22 08:44:49,126] {base_task_runner.py:95} INFO - Subtask: self.hook = airflow.hooks.hive_hooks.HiveMetastoreHook(
> [2017-02-22 08:44:49,126] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/utils/helpers.py", line 436, in __getattr__
> [2017-02-22 08:44:49,126] {base_task_runner.py:95} INFO - Subtask: raise AttributeError
> [2017-02-22 08:44:49,126] {base_task_runner.py:95} INFO - Subtask: AttributeError
> [2017-02-22 08:44:49,126] {base_task_runner.py:95} INFO - Subtask: [2017-02-22 08:44:49,125] {models.py:1441} INFO - Marking task as FAILED.
> [2017-02-22 08:44:49,163] {base_task_runner.py:95} INFO - Subtask: [2017-02-22 08:44:49,163] {models.py:1462} ERROR -
> [2017-02-22 08:44:49,163] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent call last):
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/bin/airflow", line 28, in <module>
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: args.func(args)
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/bin/cli.py", line 427, in run
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: pool=args.pool,
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/utils/db.py", line 54, in wrapper
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: result = func(*args, **kwargs)
> [2017-02-22 08:44:49,164] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: result = task_copy.execute(context=context)
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/operators/sensors.py", line 76, in execute
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: while not self.poke(context):
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/operators/sensors.py", line 375, in poke
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: self.hook = airflow.hooks.hive_hooks.HiveMetastoreHook(
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: File "/home/airflow/airflow2/lib/python2.7/site-packages/airflow/utils/helpers.py", line 436, in __getattr__
> [2017-02-22 08:44:49,165] {base_task_runner.py:95} INFO - Subtask: raise AttributeError
> {noformat}
> I've worked around the issue adding a 'import airflow.hooks.hive_hooks' in class HiveParititionSensor (because I don't really know what this is about, there surely is a real fix to be done). It fixes the issue for me.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)