[jira] [Updated] (KYLIN-5454) Downloading the async query result may cause OOM error

["Ruixuan Zhang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KYLIN-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ruixuan Zhang updated KYLIN-5454:
---------------------------------
    Attachment: 异步查询OOM.pdf

> Downloading the async query result may cause OOM error
> ------------------------------------------------------
>
>                 Key: KYLIN-5454
>                 URL: https://issues.apache.org/jira/browse/KYLIN-5454
>             Project: Kylin
>          Issue Type: Bug
>    Affects Versions: 5.0-alpha
>            Reporter: Ruixuan Zhang
>            Assignee: Ruixuan Zhang
>            Priority: Major
>             Fix For: 5.0-alpha
>
>         Attachments: 异步查询OOM.pdf
>
>
> When using {{http://host:port/kylin/api/async_query/\{query_id}/result_download?}} to get asynchronous query result, out-of-memory error might occur. The reason is the formatting process while downloading the result needs read & write, and there is batch processing rather than streaming processing. Therefore, a refactor is needed for async query. The main idea of revision is to move the formatting from downloading result from HDFS to processing & saving query result on HDFS, and use streaming read & write instead.
> The details of design are in the attached file.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)