You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Jacky Li (JIRA)" <ji...@apache.org> on 2016/10/15 05:39:20 UTC
[jira] [Updated] (CARBONDATA-318) Implement an ExternalSorter that
makes maximum usage of memory while sorting
[ https://issues.apache.org/jira/browse/CARBONDATA-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jacky Li updated CARBONDATA-318:
--------------------------------
Description:
External Sorter should sort in memory until it reach configured size, then spill to disk. It should provide following interface:
1. insertRow/insertRowBatch: insert rows into the sorter
2. getIterator: return an iterator that iterate on sorted rows
External Sorter depends on FileWriterFactory to get a FileWriter to spill data into files. FileWriterFactory should be provided by user. Multiple implementations are possible, like writing into one folder or multiple folders
was:
External Sorter should sort in memory until it reach configured size, then spill to disk. It should provide following interface:
1. insertRow/insertRowBatch: insert rows into the sorter
2. getIterator: return an iterator that iterate on sorted rows
External Sorter depends on FileWriterFactory to get a FileWriter to spill data into files. FileWriterFactory should be provided by user. Multiple implementations are possible, like writing into one folder or multiple folder
> Implement an ExternalSorter that makes maximum usage of memory while sorting
> ----------------------------------------------------------------------------
>
> Key: CARBONDATA-318
> URL: https://issues.apache.org/jira/browse/CARBONDATA-318
> Project: CarbonData
> Issue Type: Sub-task
> Reporter: Jacky Li
> Fix For: 0.2.0-incubating
>
>
> External Sorter should sort in memory until it reach configured size, then spill to disk. It should provide following interface:
> 1. insertRow/insertRowBatch: insert rows into the sorter
> 2. getIterator: return an iterator that iterate on sorted rows
> External Sorter depends on FileWriterFactory to get a FileWriter to spill data into files. FileWriterFactory should be provided by user. Multiple implementations are possible, like writing into one folder or multiple folders
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)