You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metamodel.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/08/15 20:43:45 UTC
[jira] [Commented] (METAMODEL-173) Effecient CSV writing/appending
for non-file resources
[ https://issues.apache.org/jira/browse/METAMODEL-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698397#comment-14698397 ]
ASF GitHub Bot commented on METAMODEL-173:
------------------------------------------
GitHub user kaspersorensen opened a pull request:
https://github.com/apache/metamodel/pull/40
CSV writing using Resource.write() and Resource.append()
Fixes METAMODEL-173
See also previous PR #36 which had a different approach. This approach is in my opinion better since it tries to improve the way the CsvUpdateCallback remembers the output stream (in form of a Writer) and reuses it for each line it writes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kaspersorensen/metamodel METAMODEL-173
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metamodel/pull/40.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #40
----
commit 7d21fba0a5579c96a0ab1feb5818621cfa299e9a
Author: Kasper Sørensen <i....@gmail.com>
Date: 2015-08-15T18:27:24Z
METAMODEL-173: Added Resource.append() and Resource.write() methods
implemented an abstract Resource class for reuse of common functionality
commit d1966b3756b92e3eed5943a2d1e76b949928daeb
Author: Kasper Sørensen <i....@gmail.com>
Date: 2015-08-15T18:40:00Z
METAMODEL-173: Changed CsvUpdateCallback to use Resource.write()
or Resource.append() when necesary.
----
> Effecient CSV writing/appending for non-file resources
> ------------------------------------------------------
>
> Key: METAMODEL-173
> URL: https://issues.apache.org/jira/browse/METAMODEL-173
> Project: Apache MetaModel
> Issue Type: Bug
> Reporter: Kasper Sørensen
>
> See https://github.com/apache/metamodel/pull/36
> In CsvUpdateCallback line 91-100 we have a file-specific optimization that keeps open an output stream and appends to that stream for the duration of many CSV insert operations. On other Resource implementations we use the .append call. But the .append call can be pretty expensive, so it is very ineffecient and sometimes may even provoke application errors.
> Instead we should add to the Resource interface a write() and an append() method that returns OutputStream. With those methods it becomes possible to allow the application more managed control over what is being written. Although I prefer the functional style of the current write(Action) and append(Action) methods, it seems that in practice we also need the more OO-oriented way for writing to Resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)