You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Claus Ibsen (Created) (JIRA)" <ji...@apache.org> on 2011/11/16 13:34:51 UTC
[jira] [Created] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
----------------------------------------------------------------------------------------------------------
Key: CAMEL-4687
URL: https://issues.apache.org/jira/browse/CAMEL-4687
Project: Camel
Issue Type: Task
Components: camel-core
Reporter: Claus Ibsen
Priority: Minor
Fix For: 2.9.0
We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165201#comment-13165201 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
@Claus
any idea you would like to share about the provided patch?
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Claus Ibsen (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-4687.
--------------------------------
Resolution: Fixed
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Assignee: Babak Vahdat
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Attachment: CAMEL-4687.patch
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Claus Ibsen (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-4687:
----------------------------------
Assignee: Babak Vahdat
Babak, I have granted your JIRA account karma to assign tickets.
Thanks for this patch. I should have cycles to get it into the trunk.
You like @Override :)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Assignee: Babak Vahdat
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Comment: was deleted
(was: attaching a potential patch (affected classes are of the root packages org.apache.camel.converter.* & org.apache.camel.impl.converter.*)
both in src and test folders :
- removal and fix of @SuppressWarnings usages where possible
- removal of all compiler warnings because of the unused import, raw type usage of generic types, unchecked warnings
- IOConverter buffering support for the types InputStream, OutputStream, Reader, Writer
- add of missing @Override annotation in bunch of the places (see [1])
- updated [2] accordingly to verify the buffering fix
Running of *all* test on camel-core passed on my box but *not* org.apache.camel.language.TokenPairPredicateTest, however
that seems not to be a side effect of this fix, as after reverting of *all* my local changes that test still continued to
fail, however I didn't look into it in detail as it's out of this ticket's context:
-------------------------------------------------------------------------------
Test set: org.apache.camel.language.TokenPairPredicateTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.062 sec <<< FAILURE!
testTokenPairPredicate(org.apache.camel.language.TokenPairPredicateTest) Time elapsed: 5.031 sec <<< FAILURE!
junit.framework.AssertionFailedError: File should not exists target\pair\hello.xml
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertFalse(Assert.java:34)
at org.apache.camel.language.TokenPairPredicateTest.testTokenPairPredicate(TokenPairPredicateTest.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at org.apache.camel.TestSupport.runBare(TestSupport.java:59)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:119)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at $Proxy0.invoke(Unknown Source)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:150)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
[1] http://blogs.oracle.com/ahe/entry/override_snafu
[2] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/converter/IOConverterTest.java)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164365#comment-13164365 ]
Babak Vahdat edited comment on CAMEL-4687 at 12/7/11 12:51 PM:
---------------------------------------------------------------
Another advantage of introducing & *reusing* of the 4 newly introduced IOHelper.buffered() methods (all over the places) is the fact that the following line (at least on Windows) prints 'true':
{code}
System.out.println(System.in instanceof BufferedInputStream);
{code}
Although System.in is declared to be of the type java.io.InputStream, however to me that's fair enough and makes sense to me. As the result through the reusage of these utility methods we could be more Heap friendly :-)
As an example I'm thinking of StreamConsumer having the following method signature:
{code}
private BufferedReader initializeStream() throws Exception {
// close old stream, before obtaining a new stream
IOHelper.close(inputStreamToClose);
if ("in".equals(uri)) {
inputStream = System.in;
inputStreamToClose = null;
} else if ("file".equals(uri)) {
inputStream = resolveStreamFromFile();
inputStreamToClose = inputStream;
} else if ("url".equals(uri)) {
inputStream = resolveStreamFromUrl();
inputStreamToClose = inputStream;
}
Charset charset = endpoint.getCharset();
return new BufferedReader(new InputStreamReader(inputStream, charset));
}
{code}
Indeed we do here double-buffering while streaming (at least on Windows) once through BufferedReader (which has a default buffer size of 8192 chars) and once again through inputStream (in the case of System.in) which would allocate another buffer of the size 8192 bytes. On high-load & intensive applications using Apache Camel this could bring OOME much ealier that one would expect.
The possible approach using IOHelper here could be:
{code}
new InputStreamReader(IOHelper.buffered(inputStream), charset);
{code}
was (Author: bvahdat):
Another advantage of introducing & *reusing* of the 4 newly introduced IOHelper.buffered() methods is the fact that the following line (at least on Windows) prints 'true':
{code}
System.out.println(System.in instanceof BufferedInputStream);
{code}
Although System.in is declared to be of the type java.io.InputStream, however to me that's fair enough. As the result through the reusage of these utility methods we could be more heap friendly :-)
As an example I'm thinking of StreamConsumer having the following method signature:
{code}
private BufferedReader initializeStream() throws Exception {
// close old stream, before obtaining a new stream
IOHelper.close(inputStreamToClose);
if ("in".equals(uri)) {
inputStream = System.in;
inputStreamToClose = null;
} else if ("file".equals(uri)) {
inputStream = resolveStreamFromFile();
inputStreamToClose = inputStream;
} else if ("url".equals(uri)) {
inputStream = resolveStreamFromUrl();
inputStreamToClose = inputStream;
}
Charset charset = endpoint.getCharset();
return new BufferedReader(new InputStreamReader(inputStream, charset));
}
{code}
Indeed we do here double-buffering while streaming (at least on Windows) once through BufferedReader (which has a default buffer size of 8192 chars) and once again through inputStream (in the case of System.in) which would allocate another buffer of the size 8192 bytes. On high-load & intensive applications using Apache Camel this could bring OOME much ealier that one would expect.
The possible approach using IOHelper here could be:
{code}
new InputStreamReader(IOHelper.buffered(inputStream), charset);
{code}
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Attachment: (was: CAMEL-4687.patch)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161614#comment-13161614 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
Reading again the ticket description today I assume that your main intention was to check [1] where buffering is indeed already in place, but maybe the reuse of newly introduced IOHelper.decorateXYZ() there as well could be nice.
Other than that I think naming those newly provided IOHelper methods (in patch) would be nicer. For example through *overloading* them with the *same name*, something like:
{code}
public static BufferedInputStream buffer(InputStream in)
public static BufferedOutputStream buffer(OutputStream out)
public static BufferedReader buffer(Reader reader)
public static BufferedWriter buffer(Writer writer)
{code}
There're also comments I put on [2]
[1] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/main/java/org/apache/camel/component/file/GenericFileConverter.java
[2] http://camel.465427.n5.nabble.com/svn-commit-r1209404-in-camel-trunk-camel-core-src-main-java-org-apache-camel-impl-main-java-org-apac-tp5041426p5041639.html
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Attachment: CAMEL-4687.patch
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165271#comment-13165271 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
No problem at all, just take it easy :-)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166267#comment-13166267 ]
Claus Ibsen commented on CAMEL-4687:
------------------------------------
There are some checkstyle issues.
You may want to read this page
http://camel.apache.org/building.html
And see how you can run a checkstyle test before submitting patches.
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Assignee: Babak Vahdat
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160597#comment-13160597 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
IMHO not only reading but also writing should be buffered, so that I attach a potential patch for your consideration:
(the mainly affected classes are by the root packages org.apache.camel.converter.* & org.apache.camel.impl.converter.*)
- InputStream, OutputStream, Reader and Writer io/buffering by IOConverter, StaxConverter and XmlConverter (supported through 4 new utilities by IOHelper)
- removal and fix of @SuppressWarnings usages where possible
- removal of all compiler warnings because of the unused import, raw type usage of generic types, unchecked warnings
- add of the missing @Override annotation (see [1]) in bunch of the places
- updated [2] accordingly to verify the buffering fix
- CS fixes
Running of *all* test on camel-core passed on my box but *not* org.apache.camel.language.TokenPairPredicateTest, however
that seems not to be a side effect of this fix, as after reverting of *all* changes on my workspace that test still continued
to fail, however I didn't look into it in detail as it's out of this ticket's context:
Results :
Failed tests:
testTokenPairPredicate(org.apache.camel.language.TokenPairPredicateTest): File should not exists target\pair\hello.xml
Tests run: 3945, Failures: 1, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25:14.480s
[INFO] Finished at: Thu Dec 01 03:25:48 CET 2011
[INFO] Final Memory: 11M/494M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.8:test (default-test) on project camel-core: There are test failures.
The corresponding stacktrace of the failure is:
{code}
-------------------------------------------------------------------------------
Test set: org.apache.camel.language.TokenPairPredicateTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.086 sec <<< FAILURE!
testTokenPairPredicate(org.apache.camel.language.TokenPairPredicateTest) Time elapsed: 5.086 sec <<< FAILURE!
junit.framework.AssertionFailedError: File should not exists target\pair\hello.xml
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertFalse(Assert.java:34)
at org.apache.camel.language.TokenPairPredicateTest.testTokenPairPredicate(TokenPairPredicateTest.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at org.apache.camel.TestSupport.runBare(TestSupport.java:59)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:119)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at $Proxy0.invoke(Unknown Source)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:150)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
{code}
[1] http://blogs.oracle.com/ahe/entry/override_snafu
[2] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/converter/IOConverterTest.java
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166291#comment-13166291 ]
Claus Ibsen commented on CAMEL-4687:
------------------------------------
No worries there was only about 5 or so. So I fixed them.
Maybe next time, for bigger patches, do a checkstyle run.
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Assignee: Babak Vahdat
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160185#comment-13160185 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
### Eclipse Workspace Patch 1.0
#P camel-core
Index: src/main/java/org/apache/camel/impl/converter/AsyncProcessorTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/AsyncProcessorTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/AsyncProcessorTypeConverter.java (working copy)
@@ -32,6 +32,7 @@
*/
public class AsyncProcessorTypeConverter implements TypeConverter {
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
if (value != null) {
if (type.equals(AsyncProcessor.class)) {
@@ -43,14 +44,17 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) throws NoTypeConversionAvailableException {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) throws NoTypeConversionAvailableException {
return convertTo(type, exchange, value);
}
Index: src/main/java/org/apache/camel/converter/jaxp/DomConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/jaxp/DomConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/converter/jaxp/DomConverter.java (working copy)
@@ -104,11 +104,10 @@
return Long.valueOf(s);
}
- @SuppressWarnings("unchecked")
@Converter
- public static List toList(NodeList nodeList) {
- List answer = new ArrayList();
- Iterator it = ObjectHelper.createIterator(nodeList);
+ public static List<?> toList(NodeList nodeList) {
+ List<Object> answer = new ArrayList<Object>();
+ Iterator<Object> it = ObjectHelper.createIterator(nodeList);
while (it.hasNext()) {
answer.add(it.next());
}
Index: src/main/java/org/apache/camel/impl/converter/StaticMethodFallbackTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/StaticMethodFallbackTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/StaticMethodFallbackTypeConverter.java (working copy)
@@ -45,20 +45,24 @@
return "StaticMethodFallbackTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return useExchange ? (T)ObjectHelper.invokeMethod(method, null, type, exchange, value, registry)
: (T)ObjectHelper.invokeMethod(method, null, type, value, registry);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/impl/converter/LazyLoadingTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/LazyLoadingTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/LazyLoadingTypeConverter.java (working copy)
@@ -43,7 +43,7 @@
}
@Override
- protected Object doConvertTo(final Class type, final Exchange exchange, final Object value) {
+ protected Object doConvertTo(final Class<?> type, final Exchange exchange, final Object value) {
Object answer = super.doConvertTo(type, exchange, value);
if (answer == null && !loaded.get()) {
// okay we could not convert, so try again, but load the converters up front
Index: src/test/java/org/apache/camel/converter/IOConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/IOConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/IOConverterTest.java (working copy)
@@ -16,7 +16,10 @@
*/
package org.apache.camel.converter;
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
import java.io.BufferedReader;
+import java.io.BufferedWriter;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
@@ -76,7 +79,7 @@
File file = new File("target/test/hello.txt");
OutputStream os = IOConverter.toOutputStream(file);
- assertNotNull(os);
+ assertIsInstanceOf(BufferedOutputStream.class, os);
os.close();
}
@@ -85,12 +88,12 @@
File file = new File("target/test/hello.txt");
Writer writer = IOConverter.toWriter(file, null);
- assertNotNull(writer);
+ assertIsInstanceOf(BufferedWriter.class, writer);
writer.close();
}
public void testToReader() throws Exception {
- Reader reader = IOConverter.toReader("Hello");
+ StringReader reader = IOConverter.toReader("Hello");
assertEquals("Hello", IOConverter.toString(reader));
}
@@ -99,6 +102,7 @@
exchange.setProperty(Exchange.CHARSET_NAME, ObjectHelper.getDefaultCharacterSet());
InputStream is = IOConverter.toInputStream("Hello World", exchange);
+ assertIsInstanceOf(BufferedInputStream.class, is);
assertNotNull(is);
}
@@ -108,7 +112,7 @@
BufferedReader br = new BufferedReader(new StringReader("Hello World"));
InputStream is = IOConverter.toInputStream(br, exchange);
- assertNotNull(is);
+ assertIsInstanceOf(BufferedInputStream.class, is);
}
public void testToByteArrayFile() throws Exception {
@@ -169,7 +173,7 @@
public void testToInputStreamUrl() throws Exception {
URL url = ObjectHelper.loadResourceAsURL("log4j.properties");
InputStream is = IOConverter.toInputStream(url);
- assertNotNull(is);
+ assertIsInstanceOf(BufferedInputStream.class, is);
}
public void testStringUrl() throws Exception {
Index: src/main/java/org/apache/camel/impl/converter/InstanceMethodFallbackTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/InstanceMethodFallbackTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/InstanceMethodFallbackTypeConverter.java (working copy)
@@ -48,11 +48,13 @@
return "InstanceMethodFallbackTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
Object instance = injector.newInstance();
if (instance == null) {
@@ -63,10 +65,12 @@
.invokeMethod(method, instance, type, value, registry);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/converter/IOConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/IOConverter.java (revision 1208406)
+++ src/main/java/org/apache/camel/converter/IOConverter.java (working copy)
@@ -16,8 +16,6 @@
*/
package org.apache.camel.converter;
-import java.io.BufferedInputStream;
-import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.ByteArrayInputStream;
@@ -66,12 +64,12 @@
@Converter
public static InputStream toInputStream(URL url) throws IOException {
- return url.openStream();
+ return IOHelper.decorateWithBufferedInputStream(url.openStream());
}
@Converter
public static InputStream toInputStream(File file) throws IOException {
- return new BufferedInputStream(new FileInputStream(file));
+ return IOHelper.decorateWithBufferedInputStream(new FileInputStream(file));
}
/**
@@ -84,7 +82,7 @@
@Converter
public static BufferedReader toReader(File file, Exchange exchange) throws IOException {
- return new BufferedReader(new EncodingFileReader(file, IOHelper.getCharsetName(exchange)));
+ return IOHelper.decorateWithBufferedReader(new EncodingFileReader(file, IOHelper.getCharsetName(exchange)));
}
@Converter
@@ -94,7 +92,7 @@
@Converter
public static OutputStream toOutputStream(File file) throws FileNotFoundException {
- return new BufferedOutputStream(new FileOutputStream(file));
+ return IOHelper.decorateWithBufferedOutputStream(new FileOutputStream(file));
}
/**
@@ -107,7 +105,7 @@
@Converter
public static BufferedWriter toWriter(File file, Exchange exchange) throws IOException {
- return new BufferedWriter(new EncodingFileWriter(file, IOHelper.getCharsetName(exchange)));
+ return IOHelper.decorateWithBufferedWriter(new EncodingFileWriter(file, IOHelper.getCharsetName(exchange)));
}
/**
@@ -120,7 +118,7 @@
@Converter
public static Reader toReader(InputStream in, Exchange exchange) throws IOException {
- return new InputStreamReader(in, IOHelper.getCharsetName(exchange));
+ return IOHelper.decorateWithBufferedReader(new InputStreamReader(in, IOHelper.getCharsetName(exchange)));
}
/**
@@ -133,11 +131,12 @@
@Converter
public static Writer toWriter(OutputStream out, Exchange exchange) throws IOException {
- return new OutputStreamWriter(out, IOHelper.getCharsetName(exchange));
+ return IOHelper.decorateWithBufferedWriter(new OutputStreamWriter(out, IOHelper.getCharsetName(exchange)));
}
@Converter
public static StringReader toReader(String text) {
+ // no need for buffering as the complete string input is already passed over as a whole
return new StringReader(text);
}
@@ -240,11 +239,7 @@
@Converter
public static String toString(Reader reader) throws IOException {
- if (reader instanceof BufferedReader) {
- return toString((BufferedReader)reader);
- } else {
- return toString(new BufferedReader(reader));
- }
+ return toString(IOHelper.decorateWithBufferedReader(reader));
}
@Converter
@@ -256,13 +251,10 @@
StringBuilder sb = new StringBuilder(1024);
char[] buf = new char[1024];
try {
- int len = 0;
+ int len;
// read until we reach then end which is the -1 marker
- while (len != -1) {
- len = reader.read(buf);
- if (len != -1) {
- sb.append(buf, 0, len);
- }
+ while ((len = reader.read(buf)) != -1) {
+ sb.append(buf, 0, len);
}
} finally {
IOHelper.close(reader, "reader", LOG);
@@ -312,7 +304,7 @@
@Converter
public static InputStream toInputStream(byte[] data) {
- return new ByteArrayInputStream(data);
+ return IOHelper.decorateWithBufferedInputStream(new ByteArrayInputStream(data));
}
@Converter
@@ -320,7 +312,7 @@
if (stream instanceof ObjectOutput) {
return (ObjectOutput) stream;
} else {
- return new ObjectOutputStream(stream);
+ return new ObjectOutputStream(IOHelper.decorateWithBufferedOutputStream(stream));
}
}
@@ -329,19 +321,17 @@
if (stream instanceof ObjectInput) {
return (ObjectInput) stream;
} else {
- return new ObjectInputStream(stream);
+ return new ObjectInputStream(IOHelper.decorateWithBufferedInputStream(stream));
}
}
@Converter
public static byte[] toBytes(InputStream stream) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
- try {
- IOHelper.copy(stream, bos);
- return bos.toByteArray();
- } finally {
- IOHelper.close(bos, "stream", LOG);
- }
+ IOHelper.copy(IOHelper.decorateWithBufferedInputStream(stream), bos);
+
+ // no need to close the bos ByteArrayOutputStream as per javadoc the implementation is noop
+ return bos.toByteArray();
}
@Converter
@@ -364,6 +354,7 @@
@Converter
public static InputStream toInputStream(ByteArrayOutputStream os) {
+ // no need for buffering as the complete byte array input is already passed over as a whole
return new ByteArrayInputStream(os.toByteArray());
}
Index: src/main/java/org/apache/camel/impl/converter/InstanceMethodTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/InstanceMethodTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/InstanceMethodTypeConverter.java (working copy)
@@ -49,11 +49,13 @@
return "InstanceMethodTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
Object instance = injector.newInstance();
if (instance == null) {
@@ -71,10 +73,12 @@
.invokeMethod(method, instance, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/converter/ObjectConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/ObjectConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/converter/ObjectConverter.java (working copy)
@@ -66,9 +66,8 @@
/**
* Creates an iterator over the value
*/
- @SuppressWarnings("rawtypes")
@Converter
- public static Iterator iterator(Object value) {
+ public static Iterator<?> iterator(Object value) {
return ObjectHelper.createIterator(value);
}
@@ -112,11 +111,10 @@
/**
* Returns the converted value, or null if the value is null
*/
- @SuppressWarnings("rawtypes")
@Converter
- public static Class toClass(Object value, Exchange exchange) {
+ public static Class<?> toClass(Object value, Exchange exchange) {
if (value instanceof Class) {
- return (Class) value;
+ return (Class<?>) value;
} else if (value instanceof String) {
// prefer to use class resolver API
if (exchange != null) {
Index: src/test/java/org/apache/camel/converter/ObjectHelperTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/ObjectHelperTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/ObjectHelperTest.java (working copy)
@@ -38,7 +38,7 @@
public void testArrayAsIterator() throws Exception {
String[] data = {"a", "b"};
- Iterator iter = ObjectHelper.createIterator(data);
+ Iterator<?> iter = ObjectHelper.createIterator(data);
assertTrue("should have next", iter.hasNext());
Object a = iter.next();
assertEquals("a", "a", a);
@@ -69,21 +69,21 @@
}
public void testIteratorWithComma() {
- Iterator it = ObjectHelper.createIterator("Claus,Jonathan");
+ Iterator<?> it = ObjectHelper.createIterator("Claus,Jonathan");
assertEquals("Claus", it.next());
assertEquals("Jonathan", it.next());
assertEquals(false, it.hasNext());
}
public void testIteratorWithOtherDelimiter() {
- Iterator it = ObjectHelper.createIterator("Claus#Jonathan", "#");
+ Iterator<?> it = ObjectHelper.createIterator("Claus#Jonathan", "#");
assertEquals("Claus", it.next());
assertEquals("Jonathan", it.next());
assertEquals(false, it.hasNext());
}
public void testIteratorEmpty() {
- Iterator it = ObjectHelper.createIterator("");
+ Iterator<?> it = ObjectHelper.createIterator("");
assertEquals(false, it.hasNext());
it = ObjectHelper.createIterator(" ");
Index: src/main/java/org/apache/camel/converter/jaxp/XmlConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/jaxp/XmlConverter.java (revision 1205385)
+++ src/main/java/org/apache/camel/converter/jaxp/XmlConverter.java (working copy)
@@ -28,7 +28,6 @@
import java.io.StringReader;
import java.io.StringWriter;
import java.lang.reflect.Constructor;
-import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.util.Properties;
Index: src/test/java/org/apache/camel/converter/ObjectConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/ObjectConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/ObjectConverterTest.java (working copy)
@@ -35,7 +35,7 @@
}
public void testIterator() {
- Iterator it = ObjectConverter.iterator("Claus,Jonathan");
+ Iterator<?> it = ObjectConverter.iterator("Claus,Jonathan");
assertEquals("Claus", it.next());
assertEquals("Jonathan", it.next());
assertEquals(false, it.hasNext());
Index: src/main/java/org/apache/camel/converter/CollectionConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/CollectionConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/converter/CollectionConverter.java (working copy)
@@ -123,10 +123,9 @@
* Converts an {@link Iterable} into a {@link List}
*/
@Converter
- @SuppressWarnings("unchecked")
public static <T> List<T> toList(Iterable<T> iterable) {
if (iterable instanceof List) {
- return (List) iterable;
+ return (List<T>) iterable;
}
List<T> result = new LinkedList<T>();
for (T value : iterable) {
Index: src/test/java/org/apache/camel/converter/FutureConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/FutureConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/FutureConverterTest.java (working copy)
@@ -31,14 +31,14 @@
public class FutureConverterTest extends ContextTestSupport {
public void testConvertFuture() {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
String out = context.getTypeConverter().convertTo(String.class, future);
assertEquals("Bye World", out);
}
public void testConvertMandatoryFuture() throws Exception {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
String out = context.getTypeConverter().mandatoryConvertTo(String.class, future);
assertEquals("Bye World", out);
@@ -46,7 +46,7 @@
public void testConvertMandatoryFutureWithExchange() throws Exception {
Exchange exchange = new DefaultExchange(context);
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
String out = context.getTypeConverter().mandatoryConvertTo(String.class, exchange, future);
assertEquals("Bye World", out);
@@ -54,7 +54,7 @@
public void testConvertMandatoryFutureWithExchangeFailed() throws Exception {
Exchange exchange = new DefaultExchange(context);
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
try {
context.getTypeConverter().mandatoryConvertTo(Timestamp.class, exchange, future);
@@ -66,14 +66,14 @@
public void testConvertFutureWithExchangeFailed() throws Exception {
Exchange exchange = new DefaultExchange(context);
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
Timestamp out = context.getTypeConverter().convertTo(Timestamp.class, exchange, future);
assertNull(out);
}
public void testConvertFutureCancelled() {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
future.cancel(true);
Object out = context.getTypeConverter().convertTo(String.class, future);
@@ -82,7 +82,7 @@
}
public void testConvertFutureCancelledThenOkay() {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
future.cancel(true);
Object out = context.getTypeConverter().convertTo(String.class, future);
Index: src/main/java/org/apache/camel/impl/converter/EnumTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/EnumTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/EnumTypeConverter.java (working copy)
@@ -33,6 +33,7 @@
public class EnumTypeConverter implements TypeConverter {
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
if (type.isEnum() && value != null) {
String text = value.toString();
@@ -47,16 +48,19 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
// ignore exchange
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
// ignore exchange
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
// ignore exchange
return convertTo(type, value);
Index: src/main/java/org/apache/camel/impl/converter/AnnotationTypeConverterLoader.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/AnnotationTypeConverterLoader.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/AnnotationTypeConverterLoader.java (working copy)
@@ -74,6 +74,7 @@
this.resolver = resolver;
}
+ @Override
public void load(TypeConverterRegistry registry) throws TypeConverterLoaderException {
String[] packageNames;
@@ -120,7 +121,7 @@
}
// load all the found classes into the type converter registry
- for (Class type : classes) {
+ for (Class<?> type : classes) {
if (LOG.isTraceEnabled()) {
LOG.trace("Loading converter class: {}", ObjectHelper.name(type));
}
Index: src/main/java/org/apache/camel/converter/stream/FileInputStreamCache.java
===================================================================
--- src/main/java/org/apache/camel/converter/stream/FileInputStreamCache.java (revision 1205744)
+++ src/main/java/org/apache/camel/converter/stream/FileInputStreamCache.java (working copy)
@@ -17,7 +17,6 @@
package org.apache.camel.converter.stream;
import java.io.BufferedInputStream;
-import java.io.Closeable;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
Index: src/test/java/org/apache/camel/converter/CollectionConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/CollectionConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/CollectionConverterTest.java (working copy)
@@ -32,7 +32,6 @@
/**
* Test cases for {@link CollectionConverter}
*/
-@SuppressWarnings("unchecked")
public class CollectionConverterTest extends TestCase {
private static final List<String> SMURFS = Arrays.asList("Papa smurf", "Smurfette", "Hefty smurf", "Jokey smurf");
@@ -42,13 +41,13 @@
}
public void testIterableToList() throws Exception {
- assertSmurfs(CollectionConverter.toList(new Iterable() {
- public Iterator iterator() {
+ assertSmurfs(CollectionConverter.toList(new Iterable<String>() {
+ public Iterator<String> iterator() {
return SMURFS.iterator();
}
}));
// no conversion should occur for the list itself
- assertSame(SMURFS, CollectionConverter.toList((Iterable) SMURFS));
+ assertSame(SMURFS, CollectionConverter.toList((Iterable<String>) SMURFS));
}
private void assertSmurfs(Collection<String> result) {
@@ -65,36 +64,36 @@
}
public void testToList() {
- List out = CollectionConverter.toList(SMURFS);
+ List<String> out = CollectionConverter.toList(SMURFS);
assertEquals(4, out.size());
}
public void testToSet() {
- Map map = new HashMap();
+ Map<String, String> map = new HashMap<String, String>();
map.put("foo", "bar");
- Set out = CollectionConverter.toSet(map);
+ Set<Map.Entry<String, String>> out = CollectionConverter.toSet(map);
assertEquals(1, out.size());
}
public void testToHashMap() {
- Map map = new CaseInsensitiveMap();
+ Map<String, Object> map = new CaseInsensitiveMap();
map.put("foo", "bar");
- HashMap out = CollectionConverter.toHashMap(map);
+ Map<String, Object> out = CollectionConverter.toHashMap(map);
assertEquals(1, out.size());
}
public void testToHashtable() {
- Map map = new CaseInsensitiveMap();
+ Map<String, Object> map = new CaseInsensitiveMap();
map.put("foo", "bar");
- Hashtable out = CollectionConverter.toHashtable(map);
+ Hashtable<String, Object> out = CollectionConverter.toHashtable(map);
assertEquals(1, out.size());
}
public void testToProperties() {
- Map map = new HashMap();
+ Map<Object, Object> map = new HashMap<Object, Object>();
map.put("foo", "bar");
Properties prop = CollectionConverter.toProperties(map);
Index: src/main/java/org/apache/camel/impl/converter/ArrayTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/ArrayTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/ArrayTypeConverter.java (working copy)
@@ -35,10 +35,11 @@
public class ArrayTypeConverter implements TypeConverter {
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
if (type.isArray()) {
if (value instanceof Collection) {
- Collection collection = (Collection)value;
+ Collection<?> collection = (Collection<?>)value;
Object array = Array.newInstance(type.getComponentType(), collection.size());
if (array instanceof Object[]) {
collection.toArray((Object[])array);
@@ -63,7 +64,7 @@
return (T)Arrays.asList((Object[])value);
} else if (value.getClass().isArray()) {
int size = Array.getLength(value);
- List answer = new ArrayList(size);
+ List<Object> answer = new ArrayList<Object>(size);
for (int i = 0; i < size; i++) {
answer.add(Array.get(value, i));
}
@@ -74,14 +75,17 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
Index: src/test/java/org/apache/camel/converter/jaxp/DomConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/jaxp/DomConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/jaxp/DomConverterTest.java (working copy)
@@ -62,11 +62,11 @@
Document document = context.getTypeConverter().convertTo(Document.class, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<foo><hello>Hello World</hello><bye>Bye Camel</bye></foo>");
- List list = DomConverter.toList(document.getElementsByTagName("foo"));
+ List<?> list = DomConverter.toList(document.getElementsByTagName("foo"));
assertEquals(1, list.size());
NodeList nl = assertIsInstanceOf(NodeList.class, list.get(0));
- List sub = DomConverter.toList(nl);
+ List<?> sub = DomConverter.toList(nl);
assertEquals(2, sub.size());
assertEquals("<hello>Hello World</hello>", new DomConverter().toString((NodeList) sub.get(0), null));
Index: src/main/java/org/apache/camel/impl/converter/StaticMethodTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/StaticMethodTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/StaticMethodTypeConverter.java (working copy)
@@ -41,20 +41,24 @@
return "StaticMethodTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return useExchange ? (T)ObjectHelper.invokeMethod(method, null, value, exchange)
: (T)ObjectHelper.invokeMethod(method, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/impl/converter/BaseTypeConverterRegistry.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/BaseTypeConverterRegistry.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/BaseTypeConverterRegistry.java (working copy)
@@ -39,7 +39,6 @@
import org.apache.camel.spi.TypeConverterRegistry;
import org.apache.camel.support.ServiceSupport;
import org.apache.camel.util.ObjectHelper;
-import org.apache.camel.util.ServiceHelper;
import org.apache.camel.util.StopWatch;
import org.apache.camel.util.TimeUtils;
import org.slf4j.Logger;
@@ -88,11 +87,13 @@
return typeConverterLoaders;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
if (!isRunAllowed()) {
throw new IllegalStateException(this + " is not started");
@@ -123,11 +124,13 @@
}
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) throws NoTypeConversionAvailableException {
return mandatoryConvertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) throws NoTypeConversionAvailableException {
if (!isRunAllowed()) {
throw new IllegalStateException(this + " is not started");
@@ -147,8 +150,7 @@
}
}
- @SuppressWarnings("unchecked")
- protected Object doConvertTo(final Class type, final Exchange exchange, final Object value) {
+ protected Object doConvertTo(final Class<?> type, final Exchange exchange, final Object value) {
if (log.isTraceEnabled()) {
log.trace("Converting {} -> {} with value: {}",
new Object[]{value == null ? "null" : value.getClass().getCanonicalName(),
@@ -218,7 +220,7 @@
// not found with that type then if it was a primitive type then try again with the wrapper type
if (type.isPrimitive()) {
- Class primitiveType = ObjectHelper.convertPrimitiveTypeToWrapperType(type);
+ Class<?> primitiveType = ObjectHelper.convertPrimitiveTypeToWrapperType(type);
if (primitiveType != type) {
return convertTo(primitiveType, exchange, value);
}
@@ -231,6 +233,7 @@
return Void.TYPE;
}
+ @Override
public void addTypeConverter(Class<?> toType, Class<?> fromType, TypeConverter typeConverter) {
log.trace("Adding type converter: {}", typeConverter);
TypeMapping key = new TypeMapping(toType, fromType);
@@ -249,6 +252,7 @@
}
}
+ @Override
public void addFallbackTypeConverter(TypeConverter typeConverter, boolean canPromote) {
log.trace("Adding fallback type converter: {} which can promote: {}", typeConverter, canPromote);
@@ -265,10 +269,12 @@
return typeMappings.get(key);
}
+ @Override
public Injector getInjector() {
return injector;
}
+ @Override
public void setInjector(Injector injector) {
this.injector = injector;
}
@@ -319,6 +325,7 @@
return converter;
}
+ @Override
public TypeConverter lookup(Class<?> toType, Class<?> fromType) {
return doLookup(toType, fromType, false);
}
Index: src/main/java/org/apache/camel/impl/converter/PropertyEditorTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/PropertyEditorTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/PropertyEditorTypeConverter.java (working copy)
@@ -48,6 +48,7 @@
misses.clear();
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
// We can't convert null values since we can't figure out a property
// editor for it.
@@ -107,14 +108,17 @@
}
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
Index: src/main/java/org/apache/camel/impl/converter/FutureTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/FutureTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/FutureTypeConverter.java (working copy)
@@ -53,7 +53,7 @@
if (Future.class.isAssignableFrom(value.getClass())) {
- Future future = (Future) value;
+ Future<?> future = (Future<?>) value;
if (future.isCancelled()) {
// return void to indicate its not possible to convert at this time
@@ -86,10 +86,12 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
try {
return doConvertTo(type, exchange, value);
@@ -98,10 +100,12 @@
}
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) throws NoTypeConversionAvailableException {
return mandatoryConvertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) throws NoTypeConversionAvailableException {
T answer;
try {
Index: src/main/java/org/apache/camel/impl/converter/ToStringTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/ToStringTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/ToStringTypeConverter.java (working copy)
@@ -33,6 +33,7 @@
public class ToStringTypeConverter implements TypeConverter {
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> toType, Object value) {
if (value != null) {
@@ -63,14 +64,17 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
Index: src/main/java/org/apache/camel/util/IOHelper.java
===================================================================
--- src/main/java/org/apache/camel/util/IOHelper.java (revision 1203164)
+++ src/main/java/org/apache/camel/util/IOHelper.java (working copy)
@@ -16,11 +16,17 @@
*/
package org.apache.camel.util;
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
+import java.io.Reader;
import java.io.UnsupportedEncodingException;
+import java.io.Writer;
import java.nio.charset.Charset;
import java.nio.charset.UnsupportedCharsetException;
@@ -126,6 +132,62 @@
}
/**
+ * Wraps the passed <code>in</code> into a {@link BufferedInputStream}
+ * object and returns that. If the passed <code>in</code> is already an
+ * instance of {@link BufferedInputStream} returns the same passed
+ * <code>in</code> reference as is (avoiding double wrapping).
+ *
+ * @param in the wrapee to be used for the buffering support
+ * @return the passed <code>in</code> decorated through a
+ * {@link BufferedInputStream} object as wrapper
+ */
+ public static BufferedInputStream decorateWithBufferedInputStream(InputStream in) {
+ return (in instanceof BufferedInputStream) ? (BufferedInputStream)in : new BufferedInputStream(in);
+ }
+
+ /**
+ * Wraps the passed <code>out</code> into a {@link BufferedOutputStream}
+ * object and returns that. If the passed <code>out</code> is already an
+ * instance of {@link BufferedOutputStream} returns the same passed
+ * <code>out</code> reference as is (avoiding double wrapping).
+ *
+ * @param out the wrapee to be used for the buffering support
+ * @return the passed <code>out</code> decorated through a
+ * {@link BufferedOutputStream} object as wrapper
+ */
+ public static BufferedOutputStream decorateWithBufferedOutputStream(OutputStream out) {
+ return (out instanceof BufferedOutputStream) ? (BufferedOutputStream)out : new BufferedOutputStream(out);
+ }
+
+ /**
+ * Wraps the passed <code>reader</code> into a {@link BufferedReader} object
+ * and returns that. If the passed <code>reader</code> is already an
+ * instance of {@link BufferedReader} returns the same passed
+ * <code>reader</code> reference as is (avoiding double wrapping).
+ *
+ * @param reader the wrapee to be used for the buffering support
+ * @return the passed <code>reader</code> decorated through a
+ * {@link BufferedReader} object as wrapper
+ */
+ public static BufferedReader decorateWithBufferedReader(Reader reader) {
+ return (reader instanceof BufferedReader) ? (BufferedReader)reader : new BufferedReader(reader);
+ }
+
+ /**
+ * Wraps the passed <code>writer</code> into a {@link BufferedWriter} object
+ * and returns that. If the passed <code>writer</code> is already an
+ * instance of {@link BufferedWriter} returns the same passed
+ * <code>writer</code> reference as is (avoiding double wrapping).
+ *
+ * @param writer the wrapee to be used for the buffering support
+ * @return the passed <code>writer</code> decorated through a
+ * {@link BufferedWriter} object as wrapper
+ */
+ public static BufferedWriter decorateWithBufferedWriter(Writer writer) {
+ return (writer instanceof BufferedWriter) ? (BufferedWriter)writer : new BufferedWriter(writer);
+ }
+
+ /**
* Closes the given resource if it is available, logging any closing
* exceptions to the given log
*
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Attachment: (was: CAMEL-4687.patch)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166287#comment-13166287 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
Thanks Claus :-)
I'll fix those checkstyle issues and will update the patch accordingly.
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Assignee: Babak Vahdat
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Comment: was deleted
(was: ### Eclipse Workspace Patch 1.0
#P camel-core
Index: src/main/java/org/apache/camel/impl/converter/AsyncProcessorTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/AsyncProcessorTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/AsyncProcessorTypeConverter.java (working copy)
@@ -32,6 +32,7 @@
*/
public class AsyncProcessorTypeConverter implements TypeConverter {
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
if (value != null) {
if (type.equals(AsyncProcessor.class)) {
@@ -43,14 +44,17 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) throws NoTypeConversionAvailableException {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) throws NoTypeConversionAvailableException {
return convertTo(type, exchange, value);
}
Index: src/main/java/org/apache/camel/converter/jaxp/DomConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/jaxp/DomConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/converter/jaxp/DomConverter.java (working copy)
@@ -104,11 +104,10 @@
return Long.valueOf(s);
}
- @SuppressWarnings("unchecked")
@Converter
- public static List toList(NodeList nodeList) {
- List answer = new ArrayList();
- Iterator it = ObjectHelper.createIterator(nodeList);
+ public static List<?> toList(NodeList nodeList) {
+ List<Object> answer = new ArrayList<Object>();
+ Iterator<Object> it = ObjectHelper.createIterator(nodeList);
while (it.hasNext()) {
answer.add(it.next());
}
Index: src/main/java/org/apache/camel/impl/converter/StaticMethodFallbackTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/StaticMethodFallbackTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/StaticMethodFallbackTypeConverter.java (working copy)
@@ -45,20 +45,24 @@
return "StaticMethodFallbackTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return useExchange ? (T)ObjectHelper.invokeMethod(method, null, type, exchange, value, registry)
: (T)ObjectHelper.invokeMethod(method, null, type, value, registry);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/impl/converter/LazyLoadingTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/LazyLoadingTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/LazyLoadingTypeConverter.java (working copy)
@@ -43,7 +43,7 @@
}
@Override
- protected Object doConvertTo(final Class type, final Exchange exchange, final Object value) {
+ protected Object doConvertTo(final Class<?> type, final Exchange exchange, final Object value) {
Object answer = super.doConvertTo(type, exchange, value);
if (answer == null && !loaded.get()) {
// okay we could not convert, so try again, but load the converters up front
Index: src/test/java/org/apache/camel/converter/IOConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/IOConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/IOConverterTest.java (working copy)
@@ -16,7 +16,10 @@
*/
package org.apache.camel.converter;
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
import java.io.BufferedReader;
+import java.io.BufferedWriter;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
@@ -76,7 +79,7 @@
File file = new File("target/test/hello.txt");
OutputStream os = IOConverter.toOutputStream(file);
- assertNotNull(os);
+ assertIsInstanceOf(BufferedOutputStream.class, os);
os.close();
}
@@ -85,12 +88,12 @@
File file = new File("target/test/hello.txt");
Writer writer = IOConverter.toWriter(file, null);
- assertNotNull(writer);
+ assertIsInstanceOf(BufferedWriter.class, writer);
writer.close();
}
public void testToReader() throws Exception {
- Reader reader = IOConverter.toReader("Hello");
+ StringReader reader = IOConverter.toReader("Hello");
assertEquals("Hello", IOConverter.toString(reader));
}
@@ -99,6 +102,7 @@
exchange.setProperty(Exchange.CHARSET_NAME, ObjectHelper.getDefaultCharacterSet());
InputStream is = IOConverter.toInputStream("Hello World", exchange);
+ assertIsInstanceOf(BufferedInputStream.class, is);
assertNotNull(is);
}
@@ -108,7 +112,7 @@
BufferedReader br = new BufferedReader(new StringReader("Hello World"));
InputStream is = IOConverter.toInputStream(br, exchange);
- assertNotNull(is);
+ assertIsInstanceOf(BufferedInputStream.class, is);
}
public void testToByteArrayFile() throws Exception {
@@ -169,7 +173,7 @@
public void testToInputStreamUrl() throws Exception {
URL url = ObjectHelper.loadResourceAsURL("log4j.properties");
InputStream is = IOConverter.toInputStream(url);
- assertNotNull(is);
+ assertIsInstanceOf(BufferedInputStream.class, is);
}
public void testStringUrl() throws Exception {
Index: src/main/java/org/apache/camel/impl/converter/InstanceMethodFallbackTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/InstanceMethodFallbackTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/InstanceMethodFallbackTypeConverter.java (working copy)
@@ -48,11 +48,13 @@
return "InstanceMethodFallbackTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
Object instance = injector.newInstance();
if (instance == null) {
@@ -63,10 +65,12 @@
.invokeMethod(method, instance, type, value, registry);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/converter/IOConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/IOConverter.java (revision 1208406)
+++ src/main/java/org/apache/camel/converter/IOConverter.java (working copy)
@@ -16,8 +16,6 @@
*/
package org.apache.camel.converter;
-import java.io.BufferedInputStream;
-import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.ByteArrayInputStream;
@@ -66,12 +64,12 @@
@Converter
public static InputStream toInputStream(URL url) throws IOException {
- return url.openStream();
+ return IOHelper.decorateWithBufferedInputStream(url.openStream());
}
@Converter
public static InputStream toInputStream(File file) throws IOException {
- return new BufferedInputStream(new FileInputStream(file));
+ return IOHelper.decorateWithBufferedInputStream(new FileInputStream(file));
}
/**
@@ -84,7 +82,7 @@
@Converter
public static BufferedReader toReader(File file, Exchange exchange) throws IOException {
- return new BufferedReader(new EncodingFileReader(file, IOHelper.getCharsetName(exchange)));
+ return IOHelper.decorateWithBufferedReader(new EncodingFileReader(file, IOHelper.getCharsetName(exchange)));
}
@Converter
@@ -94,7 +92,7 @@
@Converter
public static OutputStream toOutputStream(File file) throws FileNotFoundException {
- return new BufferedOutputStream(new FileOutputStream(file));
+ return IOHelper.decorateWithBufferedOutputStream(new FileOutputStream(file));
}
/**
@@ -107,7 +105,7 @@
@Converter
public static BufferedWriter toWriter(File file, Exchange exchange) throws IOException {
- return new BufferedWriter(new EncodingFileWriter(file, IOHelper.getCharsetName(exchange)));
+ return IOHelper.decorateWithBufferedWriter(new EncodingFileWriter(file, IOHelper.getCharsetName(exchange)));
}
/**
@@ -120,7 +118,7 @@
@Converter
public static Reader toReader(InputStream in, Exchange exchange) throws IOException {
- return new InputStreamReader(in, IOHelper.getCharsetName(exchange));
+ return IOHelper.decorateWithBufferedReader(new InputStreamReader(in, IOHelper.getCharsetName(exchange)));
}
/**
@@ -133,11 +131,12 @@
@Converter
public static Writer toWriter(OutputStream out, Exchange exchange) throws IOException {
- return new OutputStreamWriter(out, IOHelper.getCharsetName(exchange));
+ return IOHelper.decorateWithBufferedWriter(new OutputStreamWriter(out, IOHelper.getCharsetName(exchange)));
}
@Converter
public static StringReader toReader(String text) {
+ // no need for buffering as the complete string input is already passed over as a whole
return new StringReader(text);
}
@@ -240,11 +239,7 @@
@Converter
public static String toString(Reader reader) throws IOException {
- if (reader instanceof BufferedReader) {
- return toString((BufferedReader)reader);
- } else {
- return toString(new BufferedReader(reader));
- }
+ return toString(IOHelper.decorateWithBufferedReader(reader));
}
@Converter
@@ -256,13 +251,10 @@
StringBuilder sb = new StringBuilder(1024);
char[] buf = new char[1024];
try {
- int len = 0;
+ int len;
// read until we reach then end which is the -1 marker
- while (len != -1) {
- len = reader.read(buf);
- if (len != -1) {
- sb.append(buf, 0, len);
- }
+ while ((len = reader.read(buf)) != -1) {
+ sb.append(buf, 0, len);
}
} finally {
IOHelper.close(reader, "reader", LOG);
@@ -312,7 +304,7 @@
@Converter
public static InputStream toInputStream(byte[] data) {
- return new ByteArrayInputStream(data);
+ return IOHelper.decorateWithBufferedInputStream(new ByteArrayInputStream(data));
}
@Converter
@@ -320,7 +312,7 @@
if (stream instanceof ObjectOutput) {
return (ObjectOutput) stream;
} else {
- return new ObjectOutputStream(stream);
+ return new ObjectOutputStream(IOHelper.decorateWithBufferedOutputStream(stream));
}
}
@@ -329,19 +321,17 @@
if (stream instanceof ObjectInput) {
return (ObjectInput) stream;
} else {
- return new ObjectInputStream(stream);
+ return new ObjectInputStream(IOHelper.decorateWithBufferedInputStream(stream));
}
}
@Converter
public static byte[] toBytes(InputStream stream) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
- try {
- IOHelper.copy(stream, bos);
- return bos.toByteArray();
- } finally {
- IOHelper.close(bos, "stream", LOG);
- }
+ IOHelper.copy(IOHelper.decorateWithBufferedInputStream(stream), bos);
+
+ // no need to close the bos ByteArrayOutputStream as per javadoc the implementation is noop
+ return bos.toByteArray();
}
@Converter
@@ -364,6 +354,7 @@
@Converter
public static InputStream toInputStream(ByteArrayOutputStream os) {
+ // no need for buffering as the complete byte array input is already passed over as a whole
return new ByteArrayInputStream(os.toByteArray());
}
Index: src/main/java/org/apache/camel/impl/converter/InstanceMethodTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/InstanceMethodTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/InstanceMethodTypeConverter.java (working copy)
@@ -49,11 +49,13 @@
return "InstanceMethodTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
Object instance = injector.newInstance();
if (instance == null) {
@@ -71,10 +73,12 @@
.invokeMethod(method, instance, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/converter/ObjectConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/ObjectConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/converter/ObjectConverter.java (working copy)
@@ -66,9 +66,8 @@
/**
* Creates an iterator over the value
*/
- @SuppressWarnings("rawtypes")
@Converter
- public static Iterator iterator(Object value) {
+ public static Iterator<?> iterator(Object value) {
return ObjectHelper.createIterator(value);
}
@@ -112,11 +111,10 @@
/**
* Returns the converted value, or null if the value is null
*/
- @SuppressWarnings("rawtypes")
@Converter
- public static Class toClass(Object value, Exchange exchange) {
+ public static Class<?> toClass(Object value, Exchange exchange) {
if (value instanceof Class) {
- return (Class) value;
+ return (Class<?>) value;
} else if (value instanceof String) {
// prefer to use class resolver API
if (exchange != null) {
Index: src/test/java/org/apache/camel/converter/ObjectHelperTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/ObjectHelperTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/ObjectHelperTest.java (working copy)
@@ -38,7 +38,7 @@
public void testArrayAsIterator() throws Exception {
String[] data = {"a", "b"};
- Iterator iter = ObjectHelper.createIterator(data);
+ Iterator<?> iter = ObjectHelper.createIterator(data);
assertTrue("should have next", iter.hasNext());
Object a = iter.next();
assertEquals("a", "a", a);
@@ -69,21 +69,21 @@
}
public void testIteratorWithComma() {
- Iterator it = ObjectHelper.createIterator("Claus,Jonathan");
+ Iterator<?> it = ObjectHelper.createIterator("Claus,Jonathan");
assertEquals("Claus", it.next());
assertEquals("Jonathan", it.next());
assertEquals(false, it.hasNext());
}
public void testIteratorWithOtherDelimiter() {
- Iterator it = ObjectHelper.createIterator("Claus#Jonathan", "#");
+ Iterator<?> it = ObjectHelper.createIterator("Claus#Jonathan", "#");
assertEquals("Claus", it.next());
assertEquals("Jonathan", it.next());
assertEquals(false, it.hasNext());
}
public void testIteratorEmpty() {
- Iterator it = ObjectHelper.createIterator("");
+ Iterator<?> it = ObjectHelper.createIterator("");
assertEquals(false, it.hasNext());
it = ObjectHelper.createIterator(" ");
Index: src/main/java/org/apache/camel/converter/jaxp/XmlConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/jaxp/XmlConverter.java (revision 1205385)
+++ src/main/java/org/apache/camel/converter/jaxp/XmlConverter.java (working copy)
@@ -28,7 +28,6 @@
import java.io.StringReader;
import java.io.StringWriter;
import java.lang.reflect.Constructor;
-import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.util.Properties;
Index: src/test/java/org/apache/camel/converter/ObjectConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/ObjectConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/ObjectConverterTest.java (working copy)
@@ -35,7 +35,7 @@
}
public void testIterator() {
- Iterator it = ObjectConverter.iterator("Claus,Jonathan");
+ Iterator<?> it = ObjectConverter.iterator("Claus,Jonathan");
assertEquals("Claus", it.next());
assertEquals("Jonathan", it.next());
assertEquals(false, it.hasNext());
Index: src/main/java/org/apache/camel/converter/CollectionConverter.java
===================================================================
--- src/main/java/org/apache/camel/converter/CollectionConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/converter/CollectionConverter.java (working copy)
@@ -123,10 +123,9 @@
* Converts an {@link Iterable} into a {@link List}
*/
@Converter
- @SuppressWarnings("unchecked")
public static <T> List<T> toList(Iterable<T> iterable) {
if (iterable instanceof List) {
- return (List) iterable;
+ return (List<T>) iterable;
}
List<T> result = new LinkedList<T>();
for (T value : iterable) {
Index: src/test/java/org/apache/camel/converter/FutureConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/FutureConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/FutureConverterTest.java (working copy)
@@ -31,14 +31,14 @@
public class FutureConverterTest extends ContextTestSupport {
public void testConvertFuture() {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
String out = context.getTypeConverter().convertTo(String.class, future);
assertEquals("Bye World", out);
}
public void testConvertMandatoryFuture() throws Exception {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
String out = context.getTypeConverter().mandatoryConvertTo(String.class, future);
assertEquals("Bye World", out);
@@ -46,7 +46,7 @@
public void testConvertMandatoryFutureWithExchange() throws Exception {
Exchange exchange = new DefaultExchange(context);
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
String out = context.getTypeConverter().mandatoryConvertTo(String.class, exchange, future);
assertEquals("Bye World", out);
@@ -54,7 +54,7 @@
public void testConvertMandatoryFutureWithExchangeFailed() throws Exception {
Exchange exchange = new DefaultExchange(context);
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
try {
context.getTypeConverter().mandatoryConvertTo(Timestamp.class, exchange, future);
@@ -66,14 +66,14 @@
public void testConvertFutureWithExchangeFailed() throws Exception {
Exchange exchange = new DefaultExchange(context);
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
Timestamp out = context.getTypeConverter().convertTo(Timestamp.class, exchange, future);
assertNull(out);
}
public void testConvertFutureCancelled() {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
future.cancel(true);
Object out = context.getTypeConverter().convertTo(String.class, future);
@@ -82,7 +82,7 @@
}
public void testConvertFutureCancelledThenOkay() {
- Future future = template.asyncRequestBody("direct:foo", "Hello World");
+ Future<?> future = template.asyncRequestBody("direct:foo", "Hello World");
future.cancel(true);
Object out = context.getTypeConverter().convertTo(String.class, future);
Index: src/main/java/org/apache/camel/impl/converter/EnumTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/EnumTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/EnumTypeConverter.java (working copy)
@@ -33,6 +33,7 @@
public class EnumTypeConverter implements TypeConverter {
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
if (type.isEnum() && value != null) {
String text = value.toString();
@@ -47,16 +48,19 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
// ignore exchange
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
// ignore exchange
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
// ignore exchange
return convertTo(type, value);
Index: src/main/java/org/apache/camel/impl/converter/AnnotationTypeConverterLoader.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/AnnotationTypeConverterLoader.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/AnnotationTypeConverterLoader.java (working copy)
@@ -74,6 +74,7 @@
this.resolver = resolver;
}
+ @Override
public void load(TypeConverterRegistry registry) throws TypeConverterLoaderException {
String[] packageNames;
@@ -120,7 +121,7 @@
}
// load all the found classes into the type converter registry
- for (Class type : classes) {
+ for (Class<?> type : classes) {
if (LOG.isTraceEnabled()) {
LOG.trace("Loading converter class: {}", ObjectHelper.name(type));
}
Index: src/main/java/org/apache/camel/converter/stream/FileInputStreamCache.java
===================================================================
--- src/main/java/org/apache/camel/converter/stream/FileInputStreamCache.java (revision 1205744)
+++ src/main/java/org/apache/camel/converter/stream/FileInputStreamCache.java (working copy)
@@ -17,7 +17,6 @@
package org.apache.camel.converter.stream;
import java.io.BufferedInputStream;
-import java.io.Closeable;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
Index: src/test/java/org/apache/camel/converter/CollectionConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/CollectionConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/CollectionConverterTest.java (working copy)
@@ -32,7 +32,6 @@
/**
* Test cases for {@link CollectionConverter}
*/
-@SuppressWarnings("unchecked")
public class CollectionConverterTest extends TestCase {
private static final List<String> SMURFS = Arrays.asList("Papa smurf", "Smurfette", "Hefty smurf", "Jokey smurf");
@@ -42,13 +41,13 @@
}
public void testIterableToList() throws Exception {
- assertSmurfs(CollectionConverter.toList(new Iterable() {
- public Iterator iterator() {
+ assertSmurfs(CollectionConverter.toList(new Iterable<String>() {
+ public Iterator<String> iterator() {
return SMURFS.iterator();
}
}));
// no conversion should occur for the list itself
- assertSame(SMURFS, CollectionConverter.toList((Iterable) SMURFS));
+ assertSame(SMURFS, CollectionConverter.toList((Iterable<String>) SMURFS));
}
private void assertSmurfs(Collection<String> result) {
@@ -65,36 +64,36 @@
}
public void testToList() {
- List out = CollectionConverter.toList(SMURFS);
+ List<String> out = CollectionConverter.toList(SMURFS);
assertEquals(4, out.size());
}
public void testToSet() {
- Map map = new HashMap();
+ Map<String, String> map = new HashMap<String, String>();
map.put("foo", "bar");
- Set out = CollectionConverter.toSet(map);
+ Set<Map.Entry<String, String>> out = CollectionConverter.toSet(map);
assertEquals(1, out.size());
}
public void testToHashMap() {
- Map map = new CaseInsensitiveMap();
+ Map<String, Object> map = new CaseInsensitiveMap();
map.put("foo", "bar");
- HashMap out = CollectionConverter.toHashMap(map);
+ Map<String, Object> out = CollectionConverter.toHashMap(map);
assertEquals(1, out.size());
}
public void testToHashtable() {
- Map map = new CaseInsensitiveMap();
+ Map<String, Object> map = new CaseInsensitiveMap();
map.put("foo", "bar");
- Hashtable out = CollectionConverter.toHashtable(map);
+ Hashtable<String, Object> out = CollectionConverter.toHashtable(map);
assertEquals(1, out.size());
}
public void testToProperties() {
- Map map = new HashMap();
+ Map<Object, Object> map = new HashMap<Object, Object>();
map.put("foo", "bar");
Properties prop = CollectionConverter.toProperties(map);
Index: src/main/java/org/apache/camel/impl/converter/ArrayTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/ArrayTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/ArrayTypeConverter.java (working copy)
@@ -35,10 +35,11 @@
public class ArrayTypeConverter implements TypeConverter {
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
if (type.isArray()) {
if (value instanceof Collection) {
- Collection collection = (Collection)value;
+ Collection<?> collection = (Collection<?>)value;
Object array = Array.newInstance(type.getComponentType(), collection.size());
if (array instanceof Object[]) {
collection.toArray((Object[])array);
@@ -63,7 +64,7 @@
return (T)Arrays.asList((Object[])value);
} else if (value.getClass().isArray()) {
int size = Array.getLength(value);
- List answer = new ArrayList(size);
+ List<Object> answer = new ArrayList<Object>(size);
for (int i = 0; i < size; i++) {
answer.add(Array.get(value, i));
}
@@ -74,14 +75,17 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
Index: src/test/java/org/apache/camel/converter/jaxp/DomConverterTest.java
===================================================================
--- src/test/java/org/apache/camel/converter/jaxp/DomConverterTest.java (revision 1203164)
+++ src/test/java/org/apache/camel/converter/jaxp/DomConverterTest.java (working copy)
@@ -62,11 +62,11 @@
Document document = context.getTypeConverter().convertTo(Document.class, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<foo><hello>Hello World</hello><bye>Bye Camel</bye></foo>");
- List list = DomConverter.toList(document.getElementsByTagName("foo"));
+ List<?> list = DomConverter.toList(document.getElementsByTagName("foo"));
assertEquals(1, list.size());
NodeList nl = assertIsInstanceOf(NodeList.class, list.get(0));
- List sub = DomConverter.toList(nl);
+ List<?> sub = DomConverter.toList(nl);
assertEquals(2, sub.size());
assertEquals("<hello>Hello World</hello>", new DomConverter().toString((NodeList) sub.get(0), null));
Index: src/main/java/org/apache/camel/impl/converter/StaticMethodTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/StaticMethodTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/StaticMethodTypeConverter.java (working copy)
@@ -41,20 +41,24 @@
return "StaticMethodTypeConverter: " + method;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return useExchange ? (T)ObjectHelper.invokeMethod(method, null, value, exchange)
: (T)ObjectHelper.invokeMethod(method, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, null, value);
}
Index: src/main/java/org/apache/camel/impl/converter/BaseTypeConverterRegistry.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/BaseTypeConverterRegistry.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/BaseTypeConverterRegistry.java (working copy)
@@ -39,7 +39,6 @@
import org.apache.camel.spi.TypeConverterRegistry;
import org.apache.camel.support.ServiceSupport;
import org.apache.camel.util.ObjectHelper;
-import org.apache.camel.util.ServiceHelper;
import org.apache.camel.util.StopWatch;
import org.apache.camel.util.TimeUtils;
import org.slf4j.Logger;
@@ -88,11 +87,13 @@
return typeConverterLoaders;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
if (!isRunAllowed()) {
throw new IllegalStateException(this + " is not started");
@@ -123,11 +124,13 @@
}
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) throws NoTypeConversionAvailableException {
return mandatoryConvertTo(type, null, value);
}
@SuppressWarnings("unchecked")
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) throws NoTypeConversionAvailableException {
if (!isRunAllowed()) {
throw new IllegalStateException(this + " is not started");
@@ -147,8 +150,7 @@
}
}
- @SuppressWarnings("unchecked")
- protected Object doConvertTo(final Class type, final Exchange exchange, final Object value) {
+ protected Object doConvertTo(final Class<?> type, final Exchange exchange, final Object value) {
if (log.isTraceEnabled()) {
log.trace("Converting {} -> {} with value: {}",
new Object[]{value == null ? "null" : value.getClass().getCanonicalName(),
@@ -218,7 +220,7 @@
// not found with that type then if it was a primitive type then try again with the wrapper type
if (type.isPrimitive()) {
- Class primitiveType = ObjectHelper.convertPrimitiveTypeToWrapperType(type);
+ Class<?> primitiveType = ObjectHelper.convertPrimitiveTypeToWrapperType(type);
if (primitiveType != type) {
return convertTo(primitiveType, exchange, value);
}
@@ -231,6 +233,7 @@
return Void.TYPE;
}
+ @Override
public void addTypeConverter(Class<?> toType, Class<?> fromType, TypeConverter typeConverter) {
log.trace("Adding type converter: {}", typeConverter);
TypeMapping key = new TypeMapping(toType, fromType);
@@ -249,6 +252,7 @@
}
}
+ @Override
public void addFallbackTypeConverter(TypeConverter typeConverter, boolean canPromote) {
log.trace("Adding fallback type converter: {} which can promote: {}", typeConverter, canPromote);
@@ -265,10 +269,12 @@
return typeMappings.get(key);
}
+ @Override
public Injector getInjector() {
return injector;
}
+ @Override
public void setInjector(Injector injector) {
this.injector = injector;
}
@@ -319,6 +325,7 @@
return converter;
}
+ @Override
public TypeConverter lookup(Class<?> toType, Class<?> fromType) {
return doLookup(toType, fromType, false);
}
Index: src/main/java/org/apache/camel/impl/converter/PropertyEditorTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/PropertyEditorTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/PropertyEditorTypeConverter.java (working copy)
@@ -48,6 +48,7 @@
misses.clear();
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
// We can't convert null values since we can't figure out a property
// editor for it.
@@ -107,14 +108,17 @@
}
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
Index: src/main/java/org/apache/camel/impl/converter/FutureTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/FutureTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/FutureTypeConverter.java (working copy)
@@ -53,7 +53,7 @@
if (Future.class.isAssignableFrom(value.getClass())) {
- Future future = (Future) value;
+ Future<?> future = (Future<?>) value;
if (future.isCancelled()) {
// return void to indicate its not possible to convert at this time
@@ -86,10 +86,12 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Object value) {
return convertTo(type, null, value);
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
try {
return doConvertTo(type, exchange, value);
@@ -98,10 +100,12 @@
}
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) throws NoTypeConversionAvailableException {
return mandatoryConvertTo(type, null, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) throws NoTypeConversionAvailableException {
T answer;
try {
Index: src/main/java/org/apache/camel/impl/converter/ToStringTypeConverter.java
===================================================================
--- src/main/java/org/apache/camel/impl/converter/ToStringTypeConverter.java (revision 1203164)
+++ src/main/java/org/apache/camel/impl/converter/ToStringTypeConverter.java (working copy)
@@ -33,6 +33,7 @@
public class ToStringTypeConverter implements TypeConverter {
@SuppressWarnings("unchecked")
+ @Override
public <T> T convertTo(Class<T> toType, Object value) {
if (value != null) {
@@ -63,14 +64,17 @@
return null;
}
+ @Override
public <T> T convertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Object value) {
return convertTo(type, value);
}
+ @Override
public <T> T mandatoryConvertTo(Class<T> type, Exchange exchange, Object value) {
return convertTo(type, value);
}
Index: src/main/java/org/apache/camel/util/IOHelper.java
===================================================================
--- src/main/java/org/apache/camel/util/IOHelper.java (revision 1203164)
+++ src/main/java/org/apache/camel/util/IOHelper.java (working copy)
@@ -16,11 +16,17 @@
*/
package org.apache.camel.util;
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
+import java.io.Reader;
import java.io.UnsupportedEncodingException;
+import java.io.Writer;
import java.nio.charset.Charset;
import java.nio.charset.UnsupportedCharsetException;
@@ -126,6 +132,62 @@
}
/**
+ * Wraps the passed <code>in</code> into a {@link BufferedInputStream}
+ * object and returns that. If the passed <code>in</code> is already an
+ * instance of {@link BufferedInputStream} returns the same passed
+ * <code>in</code> reference as is (avoiding double wrapping).
+ *
+ * @param in the wrapee to be used for the buffering support
+ * @return the passed <code>in</code> decorated through a
+ * {@link BufferedInputStream} object as wrapper
+ */
+ public static BufferedInputStream decorateWithBufferedInputStream(InputStream in) {
+ return (in instanceof BufferedInputStream) ? (BufferedInputStream)in : new BufferedInputStream(in);
+ }
+
+ /**
+ * Wraps the passed <code>out</code> into a {@link BufferedOutputStream}
+ * object and returns that. If the passed <code>out</code> is already an
+ * instance of {@link BufferedOutputStream} returns the same passed
+ * <code>out</code> reference as is (avoiding double wrapping).
+ *
+ * @param out the wrapee to be used for the buffering support
+ * @return the passed <code>out</code> decorated through a
+ * {@link BufferedOutputStream} object as wrapper
+ */
+ public static BufferedOutputStream decorateWithBufferedOutputStream(OutputStream out) {
+ return (out instanceof BufferedOutputStream) ? (BufferedOutputStream)out : new BufferedOutputStream(out);
+ }
+
+ /**
+ * Wraps the passed <code>reader</code> into a {@link BufferedReader} object
+ * and returns that. If the passed <code>reader</code> is already an
+ * instance of {@link BufferedReader} returns the same passed
+ * <code>reader</code> reference as is (avoiding double wrapping).
+ *
+ * @param reader the wrapee to be used for the buffering support
+ * @return the passed <code>reader</code> decorated through a
+ * {@link BufferedReader} object as wrapper
+ */
+ public static BufferedReader decorateWithBufferedReader(Reader reader) {
+ return (reader instanceof BufferedReader) ? (BufferedReader)reader : new BufferedReader(reader);
+ }
+
+ /**
+ * Wraps the passed <code>writer</code> into a {@link BufferedWriter} object
+ * and returns that. If the passed <code>writer</code> is already an
+ * instance of {@link BufferedWriter} returns the same passed
+ * <code>writer</code> reference as is (avoiding double wrapping).
+ *
+ * @param writer the wrapee to be used for the buffering support
+ * @return the passed <code>writer</code> decorated through a
+ * {@link BufferedWriter} object as wrapper
+ */
+ public static BufferedWriter decorateWithBufferedWriter(Writer writer) {
+ return (writer instanceof BufferedWriter) ? (BufferedWriter)writer : new BufferedWriter(writer);
+ }
+
+ /**
* Closes the given resource if it is available, logging any closing
* exceptions to the given log
*
)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161614#comment-13161614 ]
Babak Vahdat edited comment on CAMEL-4687 at 12/2/11 4:15 PM:
--------------------------------------------------------------
Reading again the ticket description today I assume that your main intention was to check [1] where buffering is indeed already in place, but maybe the reuse of newly introduced IOHelper.decorateXYZ() there as well could be nice.
Other than that I think short naming of the newly provided IOHelper methods (in patch) would be nicer. For example through *overloading* them with the *same name*, something like:
{code}
public static BufferedInputStream buffered(InputStream in)
public static BufferedOutputStream buffered(OutputStream out)
public static BufferedReader buffered(Reader reader)
public static BufferedWriter buffered(Writer writer)
{code}
There're also comments I put on [2]
[1] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/main/java/org/apache/camel/component/file/GenericFileConverter.java
[2] http://camel.465427.n5.nabble.com/svn-commit-r1209404-in-camel-trunk-camel-core-src-main-java-org-apache-camel-impl-main-java-org-apac-tp5041426p5041639.html
was (Author: bvahdat):
Reading again the ticket description today I assume that your main intention was to check [1] where buffering is indeed already in place, but maybe the reuse of newly introduced IOHelper.decorateXYZ() there as well could be nice.
Other than that I think short naming of the newly provided IOHelper methods (in patch) would be nicer. For example through *overloading* them with the *same name*, something like:
{code}
public static BufferedInputStream buffer(InputStream in)
public static BufferedOutputStream buffer(OutputStream out)
public static BufferedReader buffer(Reader reader)
public static BufferedWriter buffer(Writer writer)
{code}
There're also comments I put on [2]
[1] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/main/java/org/apache/camel/component/file/GenericFileConverter.java
[2] http://camel.465427.n5.nabble.com/svn-commit-r1209404-in-camel-trunk-camel-core-src-main-java-org-apache-camel-impl-main-java-org-apac-tp5041426p5041639.html
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Comment: was deleted
(was: As uploading the patch doesn't work, I simply paste it as a comment... Sorry for that.)
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164365#comment-13164365 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
Another advantage of introducing & *reusing* of the 4 newly introduced IOHelper.buffered() methods is the fact that the following line (at least on Windows) prints 'true':
{code}
System.out.println(System.in instanceof BufferedInputStream);
{code}
Although System.in is declared to be of the type java.io.InputStream, however to me that's fair enough. As the result through the reusage of these utility methods we could be more heap friendly :-)
As an example I'm thinking of StreamConsumer having the following method signature:
{code}
private BufferedReader initializeStream() throws Exception {
// close old stream, before obtaining a new stream
IOHelper.close(inputStreamToClose);
if ("in".equals(uri)) {
inputStream = System.in;
inputStreamToClose = null;
} else if ("file".equals(uri)) {
inputStream = resolveStreamFromFile();
inputStreamToClose = inputStream;
} else if ("url".equals(uri)) {
inputStream = resolveStreamFromUrl();
inputStreamToClose = inputStream;
}
Charset charset = endpoint.getCharset();
return new BufferedReader(new InputStreamReader(inputStream, charset));
}
{code}
Indeed we do here double-buffering while streaming (at least on Windows) once through BufferedReader (which has a default buffer size of 8192 chars) and once again through inputStream (in the case of System.in) which would allocate another buffer of the size 8192 bytes. On high-load & intensive applications using Apache Camel this could bring OOME much ealier that one would expect.
The possible approach using IOHelper here could be:
{code}
new InputStreamReader(IOHelper.buffered(inputStream), charset);
{code}
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162263#comment-13162263 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
Updated the patch according to the proposal by my previous comment. The patch now also includes the javadoc typo corrections I mentioned at [1].
[1] http://camel.465427.n5.nabble.com/svn-commit-r1209404-in-camel-trunk-camel-core-src-main-java-org-apache-camel-impl-main-java-org-apac-tp5041426p5041639.html
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160182#comment-13160182 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
attaching a potential patch (affected classes are of the root packages org.apache.camel.converter.* & org.apache.camel.impl.converter.*)
both in src and test folders :
- removal and fix of @SuppressWarnings usages where possible
- removal of all compiler warnings because of the unused import, raw type usage of generic types, unchecked warnings
- IOConverter buffering support for the types InputStream, OutputStream, Reader, Writer
- add of missing @Override annotation in bunch of the places (see [1])
- updated [2] accordingly to verify the buffering fix
Running of *all* test on camel-core passed on my box but *not* org.apache.camel.language.TokenPairPredicateTest, however
that seems not to be a side effect of this fix, as after reverting of *all* my local changes that test still continued to
fail, however I didn't look into it in detail as it's out of this ticket's context:
-------------------------------------------------------------------------------
Test set: org.apache.camel.language.TokenPairPredicateTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.062 sec <<< FAILURE!
testTokenPairPredicate(org.apache.camel.language.TokenPairPredicateTest) Time elapsed: 5.031 sec <<< FAILURE!
junit.framework.AssertionFailedError: File should not exists target\pair\hello.xml
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertFalse(Assert.java:34)
at org.apache.camel.language.TokenPairPredicateTest.testTokenPairPredicate(TokenPairPredicateTest.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at org.apache.camel.TestSupport.runBare(TestSupport.java:59)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:119)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at $Proxy0.invoke(Unknown Source)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:150)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
[1] http://blogs.oracle.com/ahe/entry/override_snafu
[2] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/converter/IOConverterTest.java
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Claus Ibsen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165248#comment-13165248 ]
Claus Ibsen commented on CAMEL-4687:
------------------------------------
Thanks for the work on this Babak. I am a bit busy today.
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160184#comment-13160184 ]
Babak Vahdat commented on CAMEL-4687:
-------------------------------------
As uploading the patch doesn't work, I simply paste it as a comment... Sorry for that.
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Babak Vahdat updated CAMEL-4687:
--------------------------------
Attachment: CAMEL-4687.patch
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CAMEL-4687) Wrap FileInputStream in
BufferedInputStream in the Converter classes to ensure loading from file is
faster
Posted by "Babak Vahdat (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161614#comment-13161614 ]
Babak Vahdat edited comment on CAMEL-4687 at 12/2/11 1:22 PM:
--------------------------------------------------------------
Reading again the ticket description today I assume that your main intention was to check [1] where buffering is indeed already in place, but maybe the reuse of newly introduced IOHelper.decorateXYZ() there as well could be nice.
Other than that I think short naming of the newly provided IOHelper methods (in patch) would be nicer. For example through *overloading* them with the *same name*, something like:
{code}
public static BufferedInputStream buffer(InputStream in)
public static BufferedOutputStream buffer(OutputStream out)
public static BufferedReader buffer(Reader reader)
public static BufferedWriter buffer(Writer writer)
{code}
There're also comments I put on [2]
[1] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/main/java/org/apache/camel/component/file/GenericFileConverter.java
[2] http://camel.465427.n5.nabble.com/svn-commit-r1209404-in-camel-trunk-camel-core-src-main-java-org-apache-camel-impl-main-java-org-apac-tp5041426p5041639.html
was (Author: bvahdat):
Reading again the ticket description today I assume that your main intention was to check [1] where buffering is indeed already in place, but maybe the reuse of newly introduced IOHelper.decorateXYZ() there as well could be nice.
Other than that I think naming those newly provided IOHelper methods (in patch) would be nicer. For example through *overloading* them with the *same name*, something like:
{code}
public static BufferedInputStream buffer(InputStream in)
public static BufferedOutputStream buffer(OutputStream out)
public static BufferedReader buffer(Reader reader)
public static BufferedWriter buffer(Writer writer)
{code}
There're also comments I put on [2]
[1] https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/main/java/org/apache/camel/component/file/GenericFileConverter.java
[2] http://camel.465427.n5.nabble.com/svn-commit-r1209404-in-camel-trunk-camel-core-src-main-java-org-apache-camel-impl-main-java-org-apac-tp5041426p5041639.html
> Wrap FileInputStream in BufferedInputStream in the Converter classes to ensure loading from file is faster
> ----------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-4687
> URL: https://issues.apache.org/jira/browse/CAMEL-4687
> Project: Camel
> Issue Type: Task
> Components: camel-core
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 2.9.0
>
> Attachments: CAMEL-4687.patch
>
>
> We should check the type converters, when creating a new FileInputStream, that the stream gets wrapped in a BufferedInputStream, to ensure the file is loaded faster, as the buffered stream has an internal buffer.
> See CAMEL-4686
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira