You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/06/17 18:48:00 UTC
[jira] [Commented] (ATLAS-3198) update/delete/truncate operations
ATLAS_HOOK and ATLAS_ENTITIES improvements
[ https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865889#comment-16865889 ]
ASF subversion and git services commented on ATLAS-3198:
--------------------------------------------------------
Commit 4e1eeb4cdcd8bc0601564434176eb200b2811c2e in atlas's branch refs/heads/master from Yu-Hsin Shih
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=4e1eeb4 ]
ATLAS-3198: Stop pushing message to ATLAS_HOOK for Truncate Table/Delete data/Update data
Both ATLAS_HOOK and ATLAS_ENTITIES should stop receiving message from Truncate Table/Delete data/Update data
Signed-off-by: Sarath Subramanian <sa...@apache.org>
> update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements
> -----------------------------------------------------------------------------
>
> Key: ATLAS-3198
> URL: https://issues.apache.org/jira/browse/ATLAS-3198
> Project: Atlas
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Umesh Padashetty
> Assignee: Yu-Hsin Chang
> Priority: Critical
> Fix For: trunk
>
> Attachments: ATLAS-3198.patch, ATLAS_HOOK and ATLAS_ENTITIES messages.xlsx
>
>
> With the recent changes, we have now stopped ingesting simple insert operations since with this operation, there is no change in the metadata. Ideally, we should be doing similar improvements w.r.t. update, delete and truncate operations too.
> Currently, for update/delete/truncate, we are pushing messages in both ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided.
> If we stop ingesting updates/deletes/truncates, it will further help us reduce the burden on Kafka and save CPU cycles
> I have created and attached an excel sheet, with the list of operations and the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic, along with suitable comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)