You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hawq.apache.org by "Shivram Mani (JIRA)" <ji...@apache.org> on 2016/03/22 20:09:25 UTC

[jira] [Created] (HAWQ-577) Stream PXF metadata response

Shivram Mani created HAWQ-577:
---------------------------------

             Summary: Stream PXF metadata response 
                 Key: HAWQ-577
                 URL: https://issues.apache.org/jira/browse/HAWQ-577
             Project: Apache HAWQ
          Issue Type: Bug
          Components: PXF
            Reporter: Shivram Mani
            Assignee: Goden Yao


getMetadata api returns the metadata corresponding to the user specified pattern. There is no limit to the #of tables the pattern can correspond do and the current approach of building in memory the json object might not scale.

We needed to serialize them inside a streaming object similar to the approach used for streaming the FragmentsResponse

The same applies also for the debug function that prints metadata of all the items - if there are too many of them the StringBuilder will run out of memory. The solution in the fragments case was to print a log of one fragment at a time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)