You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Madhan Neethiraj <ma...@apache.org> on 2018/03/23 15:28:29 UTC
Review Request 66249: ATLAS-2491: updated Atlas Hive hook to use
separate
notifications for each table deleted by 'drop database cascade'
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66249/
-----------------------------------------------------------
Review request for atlas, Apoorv Naik, Ashutosh Mestry, Nixon Rodrigues, Ramesh Mani, Ruchi Solani, and Sarath Subramanian.
Bugs: ATLAS-2491
https://issues.apache.org/jira/browse/ATLAS-2491
Repository: atlas
Description
-------
- dropping a database with cascade option results in a single notificaiton that includes all deleted tables and the database. For databases containing large number of tables, Atlas server would take a longer time to process the notification and the graph transaction will get larger. To avoid this, Atlas Hive hook should generate separate notifications for each table being dropped
- also fixed the hook to avoid sending notifications with empty entity-list
Diffs
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java e6cd9d904
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java d7d6de83c
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java 2454fbbdd
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java 120985274
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java 19fe8d3ba
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java fabb00c05
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java 209909516
Diff: https://reviews.apache.org/r/66249/diff/1/
Testing
-------
- Verified the patch by dropping a database containing few thousand tables
- UTs and ITs pass successfully
Thanks,
Madhan Neethiraj
Re: Review Request 66249: ATLAS-2491: updated Atlas Hive hook to use
separate notifications for each table deleted by 'drop database
cascade'
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66249/#review199909
-----------------------------------------------------------
Ship it!
Ship It!
- Sarath Subramanian
On March 23, 2018, 8:28 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66249/
> -----------------------------------------------------------
>
> (Updated March 23, 2018, 8:28 a.m.)
>
>
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Nixon Rodrigues, Ramesh Mani, Ruchi Solani, and Sarath Subramanian.
>
>
> Bugs: ATLAS-2491
> https://issues.apache.org/jira/browse/ATLAS-2491
>
>
> Repository: atlas
>
>
> Description
> -------
>
> - dropping a database with cascade option results in a single notificaiton that includes all deleted tables and the database. For databases containing large number of tables, Atlas server would take a longer time to process the notification and the graph transaction will get larger. To avoid this, Atlas Hive hook should generate separate notifications for each table being dropped
> - also fixed the hook to avoid sending notifications with empty entity-list
>
>
> Diffs
> -----
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java e6cd9d904
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java d7d6de83c
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java 2454fbbdd
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java 120985274
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java 19fe8d3ba
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java fabb00c05
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java 209909516
>
>
> Diff: https://reviews.apache.org/r/66249/diff/1/
>
>
> Testing
> -------
>
> - Verified the patch by dropping a database containing few thousand tables
> - UTs and ITs pass successfully
>
>
> Thanks,
>
> Madhan Neethiraj
>
>