You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Pinal Shah <pi...@freestoneinfotech.com> on 2021/07/06 12:00:11 UTC
Review Request 73439: ATLAS-4347 : DSL Search : Caching Mechanism for
translated dsl queries
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73439/
-----------------------------------------------------------
Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and Sarath Subramanian.
Bugs: ATLAS-4347
https://issues.apache.org/jira/browse/ATLAS-4347
Repository: atlas
Description
-------
**Approach**
- DSL Query is converted to Tinkerpop Traversals.
- Caching traversal with respective to query saves time of translation.
- Whenever same query with same limit and offset is fired again, it will pick the translated query from cache.
**Config**
- atlas.dsl.cached.translator when 'true' uses caching mechanism (Default: true)
Diffs
-----
intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 9ef848752
repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java 724cb8244
Diff: https://reviews.apache.org/r/73439/diff/1/
Testing
-------
Thanks,
Pinal Shah
Re: Review Request 73439: ATLAS-4347 : DSL Search : Caching Mechanism
for translated dsl queries
Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73439/#review223197
-----------------------------------------------------------
Ship it!
Ship It!
- Ashutosh Mestry
On July 6, 2021, noon, Pinal Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73439/
> -----------------------------------------------------------
>
> (Updated July 6, 2021, noon)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4347
> https://issues.apache.org/jira/browse/ATLAS-4347
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Approach**
>
> - DSL Query is converted to Tinkerpop Traversals.
> - Caching traversal with respective to query saves time of translation.
> - Whenever same query with same limit and offset is fired again, it will pick the translated query from cache.
>
> **Config**
>
> - atlas.dsl.cached.translator when 'true' uses caching mechanism (Default: true)
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 9ef848752
> repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java 724cb8244
>
>
> Diff: https://reviews.apache.org/r/73439/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 73439: ATLAS-4347 : DSL Search : Caching Mechanism
for translated dsl queries
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73439/#review223207
-----------------------------------------------------------
Ship it!
Ship It!
- Sarath Subramanian
On July 6, 2021, 9:28 p.m., Pinal Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73439/
> -----------------------------------------------------------
>
> (Updated July 6, 2021, 9:28 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4347
> https://issues.apache.org/jira/browse/ATLAS-4347
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Approach**
>
> - DSL Query is converted to Tinkerpop Traversals.
> - Caching traversal with respective to query saves time of translation.
> - Whenever same query with same limit and offset is fired again, it will pick the translated query from cache.
>
> **Config**
>
> - atlas.dsl.cached.translator when 'true' uses caching mechanism (Default: true)
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 9ef848752
> repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java 724cb8244
>
>
> Diff: https://reviews.apache.org/r/73439/diff/2/
>
>
> Testing
> -------
>
> Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/718/
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 73439: ATLAS-4347 : DSL Search : Caching Mechanism
for translated dsl queries
Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.
> On July 7, 2021, 4:49 a.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java
> > Lines 169 (patched)
> > <https://reviews.apache.org/r/73439/diff/1/?file=2251111#file2251111line172>
> >
> > only the query syntax should be cached so that it can be reused not the values
> >
> > Your strategy to cache will generate 5 cache entry for same query pattern, which si not useful:
> >
> > ive_column where table.name = \"sales_fact_daily_mv1\"
> > hive_column where table.name = \"sales_fact_daily_mv2\"
> > hive_column where table.name = \"sales_fact_daily_mv3\"
> > hive_column where table.name = \"sales_fact_daily_mv4\"
> > hive_column where table.name = \"sales_fact_daily_mv5\"
Actually this approach will benefit the integrators than the end users. End users who use DSL via UI may not fire same query. The LRU cache will handle that part. Integrating applications programatically generate same queries and keep executing it at a set frequency. Caching greatly benefits those users.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73439/#review223198
-----------------------------------------------------------
On July 7, 2021, 4:28 a.m., Pinal Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73439/
> -----------------------------------------------------------
>
> (Updated July 7, 2021, 4:28 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4347
> https://issues.apache.org/jira/browse/ATLAS-4347
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Approach**
>
> - DSL Query is converted to Tinkerpop Traversals.
> - Caching traversal with respective to query saves time of translation.
> - Whenever same query with same limit and offset is fired again, it will pick the translated query from cache.
>
> **Config**
>
> - atlas.dsl.cached.translator when 'true' uses caching mechanism (Default: true)
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 9ef848752
> repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java 724cb8244
>
>
> Diff: https://reviews.apache.org/r/73439/diff/1/
>
>
> Testing
> -------
>
> Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/718/
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 73439: ATLAS-4347 : DSL Search : Caching Mechanism
for translated dsl queries
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73439/#review223198
-----------------------------------------------------------
repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java
Lines 169 (patched)
<https://reviews.apache.org/r/73439/#comment312299>
only the query syntax should be cached so that it can be reused not the values
Your strategy to cache will generate 5 cache entry for same query pattern, which si not useful:
ive_column where table.name = \"sales_fact_daily_mv1\"
hive_column where table.name = \"sales_fact_daily_mv2\"
hive_column where table.name = \"sales_fact_daily_mv3\"
hive_column where table.name = \"sales_fact_daily_mv4\"
hive_column where table.name = \"sales_fact_daily_mv5\"
- Sarath Subramanian
On July 6, 2021, 9:28 p.m., Pinal Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73439/
> -----------------------------------------------------------
>
> (Updated July 6, 2021, 9:28 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4347
> https://issues.apache.org/jira/browse/ATLAS-4347
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Approach**
>
> - DSL Query is converted to Tinkerpop Traversals.
> - Caching traversal with respective to query saves time of translation.
> - Whenever same query with same limit and offset is fired again, it will pick the translated query from cache.
>
> **Config**
>
> - atlas.dsl.cached.translator when 'true' uses caching mechanism (Default: true)
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 9ef848752
> repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java 724cb8244
>
>
> Diff: https://reviews.apache.org/r/73439/diff/1/
>
>
> Testing
> -------
>
> Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/718/
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 73439: ATLAS-4347 : DSL Search : Caching Mechanism
for translated dsl queries
Posted by Pinal Shah <pi...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73439/
-----------------------------------------------------------
(Updated July 7, 2021, 4:28 a.m.)
Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, and Sarath Subramanian.
Bugs: ATLAS-4347
https://issues.apache.org/jira/browse/ATLAS-4347
Repository: atlas
Description
-------
**Approach**
- DSL Query is converted to Tinkerpop Traversals.
- Caching traversal with respective to query saves time of translation.
- Whenever same query with same limit and offset is fired again, it will pick the translated query from cache.
**Config**
- atlas.dsl.cached.translator when 'true' uses caching mechanism (Default: true)
Diffs
-----
intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 9ef848752
repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java 724cb8244
Diff: https://reviews.apache.org/r/73439/diff/1/
Testing (updated)
-------
Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/718/
Thanks,
Pinal Shah