You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Ashutosh Mestry (Jira)" <ji...@apache.org> on 2020/02/06 23:44:00 UTC
[jira] [Resolved] (ATLAS-3608) Hive Bridge: Hive Metastore: Alter
Table Query Not Handled Correctly
[ https://issues.apache.org/jira/browse/ATLAS-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Mestry resolved ATLAS-3608.
------------------------------------
Resolution: Fixed
> Hive Bridge: Hive Metastore: Alter Table Query Not Handled Correctly
> --------------------------------------------------------------------
>
> Key: ATLAS-3608
> URL: https://issues.apache.org/jira/browse/ATLAS-3608
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Priority: Major
> Fix For: trunk, 2.0.0
>
> Attachments: ATLAS-3608-Incorrect-processing-of-alter-table.patch
>
>
> *Background*
> DDL queries in Impala are processed via _Hive Metastore_ bridge.
>
> *Steps to Duplicate*
> Keep HMS logs in view. Depending on installation, they can be found at this location: _/var/log/hive/hadoop-cmf-HIVE-1-HIVEMETASTORE-<host name>.log.out_
> From Impala:
> - Run _impala-shell_
> * Run _create database stocks; use stocks; create table daily (dt string, open string, high string); create view daily_rpt as select * from daily; create external table weekly (dt string, open string, high string);_
> * Note within Atlas that the new entities for _stocks, daily, daily_rpt_ and _weekly_ have been created. Note the columns in _weekly_ table.
> * From _impala-shell,_ run _alter table weekly add columns ( newCol string_);
> _Expected results_
> * HMS logs should not show _NullPointerException_.
> * Atlas should show the table weekly with the newCol column.
>
> _Observed results_:
> HMS logs show _NullPointerExcetion_ from Atlas hook.
> New entity _newCol_ is not seen within Atlas.
> *Root cause*
> When assessing the incoming event to determine the type of alter, Atlas uses table parameter. The recent build has a new parameter for timestamp: _last_modified_time_ _transient_lastDdlTime_. This results in incorrect assessment. Hence the alter event is incorrectly processed, thereby causing an exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)