You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Sarath Kumar Subramanian <sa...@gmail.com> on 2016/08/30 06:20:42 UTC
Review Request 51515: ATLAS-772 : New attribute has been added to the
hive_column type to capture the position of hive columns for sorting
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51515/
-----------------------------------------------------------
Review request for atlas, Madhan Neethiraj and Suma Shivaprasad.
Bugs: ATLAS-772
https://issues.apache.org/jira/browse/ATLAS-772
Repository: atlas
Description
-------
When a schema query is issued against a hive table, the hive_column entities are retrieved from the titan database. Each hive_column entity is a vertex in titan and currently titan doesn't store any column order position in its vertex attributes. The hive_columns are retrieved in the sort order of the Vertex ID which is not same as the Hive column order position. The fix attached is to introduce a new "position" attribute in hive_column type, which maintains the column order position. This column order position can be consumed by any clients - including UI to display the columns in the original position/order.
Diffs
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 1f13d74
addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataModelGenerator.java b308cc9
Diff: https://reviews.apache.org/r/51515/diff/
Testing
-------
Tested the changes with hive column operations \u2013 Add new column, Drop column, Rename column, Move column to first or at a position.
Thanks,
Sarath Kumar Subramanian
Re: Review Request 51515: ATLAS-772 : New attribute has been added to
the
hive_column type to capture the position of hive columns for sorting
Posted by Shwetha GS <ss...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51515/#review148723
-----------------------------------------------------------
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java (line 588)
<https://reviews.apache.org/r/51515/#comment216225>
Can you add tests that column position is set correctly in create table and alter table column reorder? You can add extra validations in existing tests as well
- Shwetha GS
On Aug. 30, 2016, 6:20 a.m., Sarath Kumar Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51515/
> -----------------------------------------------------------
>
> (Updated Aug. 30, 2016, 6:20 a.m.)
>
>
> Review request for atlas, Madhan Neethiraj and Suma Shivaprasad.
>
>
> Bugs: ATLAS-772
> https://issues.apache.org/jira/browse/ATLAS-772
>
>
> Repository: atlas
>
>
> Description
> -------
>
> When a schema query is issued against a hive table, the hive_column entities are retrieved from the titan database. Each hive_column entity is a vertex in titan and currently titan doesn't store any column order position in its vertex attributes. The hive_columns are retrieved in the sort order of the Vertex ID which is not same as the Hive column order position. The fix attached is to introduce a new "position" attribute in hive_column type, which maintains the column order position. This column order position can be consumed by any clients - including UI to display the columns in the original position/order.
>
>
> Diffs
> -----
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 1f13d74
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataModelGenerator.java b308cc9
>
> Diff: https://reviews.apache.org/r/51515/diff/
>
>
> Testing
> -------
>
> Tested the changes with hive column operations \u2013 Add new column, Drop column, Rename column, Move column to first or at a position.
>
>
> Thanks,
>
> Sarath Kumar Subramanian
>
>