[jira] [Commented] (ASTERIXDB-3029) Allow single-pass nested value traversal

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ASTERIXDB-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698125#comment-17698125 ] 

ASF subversion and git services commented on ASTERIXDB-3029:
------------------------------------------------------------

Commit 4b5a6a7f071d1540d913d40ff1c24d9d72cabfd8 in asterixdb's branch refs/heads/master from Wail Alkowaileet
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=4b5a6a7f07 ]

[ASTERIXDB-3029][RT] Introduce LazyVisitablePointable

- user mode changes: no
- storage format changes: no
- interface changes: no

Details:
- The lazy visitable pointable allows for traversing a record and
  processing it in a single pass compared to the IVisitablePointable,
  which requires two passes - one for calling set() and another when
  calling accept().
- Does not require writing the nested values into temporary buffers.
- However, it is suitable when traversing the records in DFS.
- Traversing and processing records efficiently is crucial for
  ASTERIXDB-3002 to infer the schema and covert rows into columns.

Change-Id: Iaf2c827d80a2a7e77f5f6eb5438cea417387e132
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17412
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Wail Alkowaileet <wa...@gmail.com>
Reviewed-by: Ali Alsuliman <al...@gmail.com>


> Allow single-pass nested value traversal
> ----------------------------------------
>
>                 Key: ASTERIXDB-3029
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-3029
>             Project: Apache AsterixDB
>          Issue Type: Improvement
>          Components: RT - Runtime
>    Affects Versions: 0.9.9
>            Reporter: Wail Y. Alkowaileet
>            Assignee: Wail Y. Alkowaileet
>            Priority: Major
>             Fix For: 0.9.9
>
>
> Currently, AsterixDB uses IVisitablePointable to allow functions (e.g., IPrintVisitor) to traverse and process (print for IPrintVisitor) nested values. The problem is that IVisitablePointable eagerly materializes all nested values into temporary buffers before a nested value could be processed, which is expensive.
> For the columnar format, AsterixDB needs to infer the schema and columnize the rows during data ingestion, which requires traversing nested values. Using IVisitablePointable in this case could hinder the ingestion rate drastically.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)