You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@streampipes.apache.org by "Philipp Zehnder (Jira)" <ji...@apache.org> on 2021/03/08 08:46:00 UTC

[jira] [Closed] (STREAMPIPES-310) Improve export speed of data download

     [ https://issues.apache.org/jira/browse/STREAMPIPES-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philipp Zehnder closed STREAMPIPES-310.
---------------------------------------
    Resolution: Fixed

It seems like the performance problem came from reading the data from the influxdb.

This is done in pages and the results are then streamed to the client. I increased the page size to load more data from the db in one step. The drawback of this solution is that it takes a bit longer till the user gets a first feedback in the UI, however it significantly increased the export speed or big files.

> Improve export speed of data download 
> --------------------------------------
>
>                 Key: STREAMPIPES-310
>                 URL: https://issues.apache.org/jira/browse/STREAMPIPES-310
>             Project: StreamPipes
>          Issue Type: Improvement
>          Components: Data Lake
>            Reporter: Philipp Zehnder
>            Assignee: Philipp Zehnder
>            Priority: Major
>             Fix For: 0.68.0
>
>
> The current performance for the data export in the data explorer is poor.
> For small files it is ok, but for big files it is very slow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)