[jira] [Assigned] (ATLAS-4501) Table 'Allow list' with a default deny to load only a subset of tables

["Ashutosh Mestry (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ATLAS-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Mestry reassigned ATLAS-4501:
--------------------------------------

    Assignee: Radhika Kundam

> Table 'Allow list' with a default deny to load only a subset of tables
> ----------------------------------------------------------------------
>
>                 Key: ATLAS-4501
>                 URL: https://issues.apache.org/jira/browse/ATLAS-4501
>             Project: Atlas
>          Issue Type: New Feature
>          Components:  atlas-core, atlas-intg, hive-integration
>            Reporter: Adriano
>            Assignee: Radhika Kundam
>            Priority: Major
>
> There are some huge environments where the warehouse has a thousand databases and hundred thousand tables with many columns and most of them are dropped, created, updated at a fast pace. In these environments, the Atlas processing time can slow down increasing the backlog as it starts moving slower than the changes in the warehouse and the {{prune.pattern}} e/o {{ignore.pattern}} it is not suitable.
> It will be nice to have the opportunity to have a default deny behaviour for all the tables and then to 'allow' the import of a subset of tables specified in a parameter regex (in order to process only some important tables): basically that works in the opposite way to the {{prune.pattern}} and {{ignore.pattern.}}
> As far as I know, there is a similar feature for S3 and ADLS but not for hive.
> If this is the case, will be nice to get the feature onboarded in your backlog.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)