You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Amit Gupta (JIRA)" <ji...@apache.org> on 2012/10/15 22:31:02 UTC
[jira] [Created] (IMAGING-95) Some tiff processing takes very long
Amit Gupta created IMAGING-95:
---------------------------------
Summary: Some tiff processing takes very long
Key: IMAGING-95
URL: https://issues.apache.org/jira/browse/IMAGING-95
Project: Commons Imaging
Issue Type: Bug
Components: Format: TIFF
Affects Versions: 1.0
Reporter: Amit Gupta
org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
In case of a tiff, which has large number of strips, skip is repeatedly called in use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IMAGING-95) Some tiff processing takes very long
Posted by "Amit Gupta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amit Gupta updated IMAGING-95:
------------------------------
Attachment: tiff_perf_fix.patch
patch for skipping array copy for skip operation
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called in use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (IMAGING-95) Some tiff processing takes
very long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479190#comment-13479190 ]
Gary Lucas edited comment on IMAGING-95 at 10/18/12 6:01 PM:
-------------------------------------------------------------
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
was (Author: gwlucas):
{{monospaced}}
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (IMAGING-95) Some tiff processing takes
very long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479190#comment-13479190 ]
Gary Lucas edited comment on IMAGING-95 at 10/18/12 8:24 PM:
-------------------------------------------------------------
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
{code}
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
{code}
was (Author: gwlucas):
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
{monospaced}
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
{monospaced}
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IMAGING-95) Some tiff processing takes very
long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479142#comment-13479142 ]
Gary Lucas commented on IMAGING-95:
-----------------------------------
I'm not part of the Apache Commons Imaging project team, but I am interested in this issue... Few people would welcome a speed enhancement for TIFF images with more enthusiasm than I :-)
I tried your patch with a couple of test files I have and didn't see much different in measured processing time. Unfortunately, I've got all sorts of security software running on my computer and it makes for a noisy test environment, so I can't be sure. So I have a couple of questions:
a) Are there specific TIFF formats where you've observed an improvement?
b) Have you tried running before-and-after tests using the ApacheImagingSpeedAndMemoryTest.java application?
The ApacheImagingSpeedAndMemory.java class is in the Commons Imaging code distribution under the path src\test\java\org\apache\commons\imaging\examples There are detailed instructions for using it in the comments inside the code. It is designed to give reasonably unambiguous timing information about how long it takes to load an image (it is coded for TIFF images, but can be edited to process other formats).
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IMAGING-95) Some tiff processing takes very
long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479190#comment-13479190 ]
Gary Lucas commented on IMAGING-95:
-----------------------------------
{{monospaced}}
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IMAGING-95) Some tiff processing takes very long
Posted by "Amit Gupta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amit Gupta updated IMAGING-95:
------------------------------
Attachment: tiff_perf_fix2.patch
patch 2, cleaned the code
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch, tiff_perf_fix.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IMAGING-95) Some tiff processing takes very long
Posted by "Amit Gupta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amit Gupta updated IMAGING-95:
------------------------------
Attachment: (was: tiff_perf_fix.patch)
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (IMAGING-95) Some tiff processing takes
very long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479190#comment-13479190 ]
Gary Lucas edited comment on IMAGING-95 at 10/18/12 8:17 PM:
-------------------------------------------------------------
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
{monospaced}
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
was (Author: gwlucas):
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (IMAGING-95) Some tiff processing takes
very long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479142#comment-13479142 ]
Gary Lucas edited comment on IMAGING-95 at 10/18/12 5:06 PM:
-------------------------------------------------------------
I'm not part of the Apache Commons Imaging project team, but I am interested in this issue... Few people would welcome a speed enhancement for TIFF images with more enthusiasm than I :-)
I tried your patch with a couple of test files I have and didn't see much difference in measured processing time. Unfortunately, I've got all sorts of security software running on my computer and it makes for a noisy test environment, so I can't be sure. So I have a couple of questions:
a) Are there specific TIFF formats where you've observed an improvement?
b) Have you tried running before-and-after tests using the ApacheImagingSpeedAndMemoryTest.java application?
The ApacheImagingSpeedAndMemory.java class is in the Commons Imaging code distribution under the path src\test\java\org\apache\commons\imaging\examples There are detailed instructions for using it in the comments inside the code. It is designed to give reasonably unambiguous timing information about how long it takes to load an image (it is coded for TIFF images, but can be edited to process other formats).
That being said, I would encourage you to include test statistics and more specific details about your test procedures when posting about performance issues. Speaking from personal experience, doing so does a lot to add credibility to your work (and who among us doesn't need more of that ?).
was (Author: gwlucas):
I'm not part of the Apache Commons Imaging project team, but I am interested in this issue... Few people would welcome a speed enhancement for TIFF images with more enthusiasm than I :-)
I tried your patch with a couple of test files I have and didn't see much difference in measured processing time. Unfortunately, I've got all sorts of security software running on my computer and it makes for a noisy test environment, so I can't be sure. So I have a couple of questions:
a) Are there specific TIFF formats where you've observed an improvement?
b) Have you tried running before-and-after tests using the ApacheImagingSpeedAndMemoryTest.java application?
The ApacheImagingSpeedAndMemory.java class is in the Commons Imaging code distribution under the path src\test\java\org\apache\commons\imaging\examples There are detailed instructions for using it in the comments inside the code. It is designed to give reasonably unambiguous timing information about how long it takes to load an image (it is coded for TIFF images, but can be edited to process other formats).
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (IMAGING-95) Some tiff processing takes
very long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479190#comment-13479190 ]
Gary Lucas edited comment on IMAGING-95 at 10/18/12 8:18 PM:
-------------------------------------------------------------
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
{monospaced}
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
{monospaced}
was (Author: gwlucas):
Well, I took another look at this, and it appears that my first tests weren't accessing the ByteSourceInputStream, but were using the ByteSourceFile. So I tried it again, following my own suggestion about using the ApacheImagingSpeedAndMemoryTest application.
The results loading a 3600 by 1800 pixel image
using current version: 763.8 milliseconds
using your version: 176.4 milliseconds
So it looks like you're on the right track. Strictly as an aside,
I also tried testing with the ByteSourceFile and got a load time of
98 milliseconds.
{monospaced}
Using ByteSourceInputStream -------------------------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
787.793 0.000 -- 63.075 92.191
859.587 0.000 -- 63.070 92.191
772.965 772.965 -- 62.894 84.473
651.912 712.439 -- 63.058 92.203
851.362 758.747 -- 63.061 92.203
818.632 773.718 -- 62.890 84.484
659.307 750.836 -- 63.058 92.203
911.201 777.563 -- 63.061 92.203
781.244 778.089 -- 62.890 84.484
663.920 763.818 -- 63.058 92.203
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
295.915 0.000 -- 63.073 92.188
193.905 0.000 -- 63.068 92.188
191.630 191.630 -- 62.892 84.473
214.639 203.134 -- 63.056 92.199
211.340 205.870 -- 63.059 92.199
182.537 200.036 -- 62.888 84.480
175.037 195.037 -- 63.056 92.199
166.024 190.201 -- 63.059 92.199
149.707 184.416 -- 62.888 84.480
176.374 183.411 -- 63.057 92.199
Using ByteSourceFile instead of ByteSourceInputStream ------------------
Processing file: BlueMarble.TIFF
image size: 3600 by 1800
time to load image -- memory
time ms avg ms -- used mb total mb
184.074 0.000 -- 26.417 40.285
90.738 0.000 -- 29.509 51.410
88.949 88.949 -- 30.485 47.816
104.790 96.869 -- 26.166 40.289
99.904 97.881 -- 26.084 40.289
98.153 97.949 -- 26.084 40.289
98.774 98.114 -- 26.084 40.289
98.111 98.113 -- 26.084 40.289
97.828 98.073 -- 26.084 40.289
98.742 98.156 -- 26.084 40.289
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (IMAGING-95) Some tiff processing takes very long
Posted by "Damjan Jovanovic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damjan Jovanovic resolved IMAGING-95.
-------------------------------------
Resolution: Fixed
Fix Version/s: 1.0
Fixed. Thank you for your patch!
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Fix For: 1.0
>
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IMAGING-95) Some tiff processing takes very
long
Posted by "Amit Gupta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476869#comment-13476869 ]
Amit Gupta commented on IMAGING-95:
-----------------------------------
somehow, I can not see check box to grant license for inclusion.
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IMAGING-95) Some tiff processing takes very long
Posted by "Amit Gupta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amit Gupta updated IMAGING-95:
------------------------------
Description:
org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
was:
org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
In case of a tiff, which has large number of strips, skip is repeatedly called in use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (IMAGING-95) Some tiff processing takes
very long
Posted by "Gary Lucas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IMAGING-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479142#comment-13479142 ]
Gary Lucas edited comment on IMAGING-95 at 10/18/12 5:00 PM:
-------------------------------------------------------------
I'm not part of the Apache Commons Imaging project team, but I am interested in this issue... Few people would welcome a speed enhancement for TIFF images with more enthusiasm than I :-)
I tried your patch with a couple of test files I have and didn't see much difference in measured processing time. Unfortunately, I've got all sorts of security software running on my computer and it makes for a noisy test environment, so I can't be sure. So I have a couple of questions:
a) Are there specific TIFF formats where you've observed an improvement?
b) Have you tried running before-and-after tests using the ApacheImagingSpeedAndMemoryTest.java application?
The ApacheImagingSpeedAndMemory.java class is in the Commons Imaging code distribution under the path src\test\java\org\apache\commons\imaging\examples There are detailed instructions for using it in the comments inside the code. It is designed to give reasonably unambiguous timing information about how long it takes to load an image (it is coded for TIFF images, but can be edited to process other formats).
was (Author: gwlucas):
I'm not part of the Apache Commons Imaging project team, but I am interested in this issue... Few people would welcome a speed enhancement for TIFF images with more enthusiasm than I :-)
I tried your patch with a couple of test files I have and didn't see much different in measured processing time. Unfortunately, I've got all sorts of security software running on my computer and it makes for a noisy test environment, so I can't be sure. So I have a couple of questions:
a) Are there specific TIFF formats where you've observed an improvement?
b) Have you tried running before-and-after tests using the ApacheImagingSpeedAndMemoryTest.java application?
The ApacheImagingSpeedAndMemory.java class is in the Commons Imaging code distribution under the path src\test\java\org\apache\commons\imaging\examples There are detailed instructions for using it in the comments inside the code. It is designed to give reasonably unambiguous timing information about how long it takes to load an image (it is coded for TIFF images, but can be edited to process other formats).
> Some tiff processing takes very long
> ------------------------------------
>
> Key: IMAGING-95
> URL: https://issues.apache.org/jira/browse/IMAGING-95
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: TIFF
> Affects Versions: 1.0
> Reporter: Amit Gupta
> Attachments: tiff_perf_fix2.patch
>
>
> org.apache.commons.imaging.formats.tiff.TiffReader.getTiffRawImageData(ByteSource, TiffDirectory) 226635 1
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.getBlock(int, int) 226588 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int) 226526 5616
> org.apache.commons.imaging.common.BinaryFileFunctions.skipBytes(InputStream, int, String) 226526 5616
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheReadingInputStream.read(byte[], int, int) 226526 188656860
> org.apache.commons.imaging.common.bytesource.ByteSourceInputStream$CacheBlock.getNext() 64581 188651244
> skip bytes is being called repeatedly again and again, last column is invocation count in one call tree. Second column is total number of time taken by that method in that call tree..
> and skip method is not overridden org.apache.commons.imaging.common.bytesource.ByteSourceInputStream.CacheReadingInputStream and default implementation of InputStream tries to use read method (which is overridden in CacheReadingInputStream) to skip.
> In case of a tiff, which has large number of strips, skip is repeatedly called and use of read is inefficient as it tried to do a System.arraycopy. array copy is not needed in case of skip operation, as the bytes were already read in block/cached, we can simply jump the pointer (block by block)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira