You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metamodel.apache.org by ClaudiaPHI <gi...@git.apache.org> on 2016/06/08 14:50:11 UTC
[GitHub] metamodel pull request #109: Improving writing to an excel file
GitHub user ClaudiaPHI opened a pull request:
https://github.com/apache/metamodel/pull/109
Improving writing to an excel file
In DataCleaner we have a performance issue when trying to write an Excel file: https://github.com/datacleaner/DataCleaner/issues/1370
I removed the workaround to "first write to a temp file to avoid that workbook source is the same as the target (will cause read+write cyclic overflow)"
The "InMemoryResource" is not needed anymore after the change I have made.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ClaudiaPHI/metamodel feature/improve-excel-writing
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metamodel/pull/109.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #109
----
commit 543370fbf73b780920e46a892300d4947e5e7ce6
Author: claudiap <cl...@humaninference.com>
Date: 2016-06-08T14:43:27Z
improved writing to excel. removed the InMemoryResource
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #109: Improving writing to an excel file
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on a diff in the pull request:
https://github.com/apache/metamodel/pull/109#discussion_r66380163
--- Diff: full/src/main/java/org/apache/metamodel/DataContextFactory.java ---
@@ -571,6 +571,8 @@ public static UpdateableDataContext createMongoDbDataContext(String hostname, In
throw (RuntimeException) e;
}
throw new IllegalStateException(e);
+ } finally {
+ mongoClient.close();
--- End diff --
It should not be closed. That would leave the created DataContext unusable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #109: Improving writing to an excel file
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on a diff in the pull request:
https://github.com/apache/metamodel/pull/109#discussion_r66380131
--- Diff: excel/src/main/java/org/apache/metamodel/excel/ExcelUtils.java ---
@@ -153,20 +155,15 @@ public static Workbook readWorkbook(ExcelDataContext dataContext) {
* @param wb
*/
public static void writeAndCloseWorkbook(ExcelDataContext dataContext, final Workbook wb) {
- // first write to a temp file to avoid that workbook source is the same
- // as the target (will cause read+write cyclic overflow)
--- End diff --
I think the gist of this is just that you're disregarding this comment now? That would cause an old bug to reoccur.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #109: Improving writing to an excel file
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on a diff in the pull request:
https://github.com/apache/metamodel/pull/109#discussion_r66380066
--- Diff: excel/src/main/java/org/apache/metamodel/excel/ExcelUtils.java ---
@@ -108,7 +108,9 @@ public static Workbook readWorkbook(Resource resource) {
if (resource instanceof FileResource) {
final File file = ((FileResource) resource).getFile();
try {
- return WorkbookFactory.create(file);
+ final FileInputStream inputStream = new FileInputStream(file);
+ final Workbook workbook = WorkbookFactory.create(inputStream);
--- End diff --
I'm pretty sure this will harm performance rather than improve it. WorkbookFactory has a specific create-method that takes a File because that enables it to use RandomAccessFile when working with it, rather than having to sequence through it like an input stream.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #109: Improving writing to an excel file
Posted by ClaudiaPHI <gi...@git.apache.org>.
Github user ClaudiaPHI closed the pull request at:
https://github.com/apache/metamodel/pull/109
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel issue #109: Improving writing to an excel file
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on the issue:
https://github.com/apache/metamodel/pull/109
-1, I don't think these changes are viable. Please see review remark.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---