[jira] [Commented] (KYLIN-5454) Downloading the async query result may cause OOM error

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695558#comment-17695558 ] 

ASF subversion and git services commented on KYLIN-5454:
--------------------------------------------------------

Commit 968a9ca4466ff2eb7e72a0b72e5c1626672fba23 in kylin's branch refs/heads/doc5.0 from DorrisRuixuanZhang
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=968a9ca446 ]

KYLIN-5454 update async query api


> Downloading the async query result may cause OOM error
> ------------------------------------------------------
>
>                 Key: KYLIN-5454
>                 URL: https://issues.apache.org/jira/browse/KYLIN-5454
>             Project: Kylin
>          Issue Type: Bug
>    Affects Versions: 5.0-alpha
>            Reporter: Ruixuan Zhang
>            Assignee: Ruixuan Zhang
>            Priority: Major
>             Fix For: 5.0-alpha
>
>         Attachments: Dev Design - Async Query may cause OOM.pdf
>
>
> When using {{http://host:port/kylin/api/async_query/\{query_id}/result_download?}} to get asynchronous query result, out-of-memory error might occur. The reason is the formatting process while downloading the result needs read & write, and there is batch processing rather than streaming processing. Therefore, a refactor is needed for async query. The main idea of revision is to move the formatting from downloading result from HDFS to processing & saving query result on HDFS, and use streaming read & write instead.
> The details of design are in the attached file.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)