You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Patrick Herber <pa...@gmail.com> on 2009/09/01 12:33:23 UTC
[compress] add a memory efficient stream compress InputStream - e.g.
a "DeflaterInputStream"
Hello!
I've the same issue as described in
https://issues.apache.org/jira/browse/COMPRESS-48
My web application recieves an InputStream (an uploaded large file)
which I need to save compressed in the DB.
Currently (since the file could be really big) I save it GZIP-ed into a
Temp File and then I open a new InputStream to that file. However this
process take up more than 60% of the overall method execution:
tempFile = File.createTempFile(Long.toString(getId()), null);
tempFile.deleteOnExit();
GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream(tempFile));
IOUtils.copy(is, gos);
is = new FileInputStream(tempFile);
stmt.setBinaryStream(3, is, -1);
stmt.executeUpdate();
The issue is marked as fixed but I didn't find how/where.
Could you kindly give me a sample how to do it?
Thanks a lot for your help
Best regards
Patrick
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: log4j NTEventLogAppender.dll for windows 64 bit
Posted by Niall Pemberton <ni...@gmail.com>.
On Tue, Sep 1, 2009 at 10:27 PM, Trin
Chavalittumrong<tr...@microsoft.com> wrote:
> Hi All,
>
> The NTEVentLogAPpender.dll that comes with log4j seems to be working with Windows 32-bit only. Has anyone had any solutions for this problem in Windows 64-bit?
Wrong list, try http://logging.apache.org/mail-lists.html
Niall
> Thanks,
> Trin
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
log4j NTEventLogAppender.dll for windows 64 bit
Posted by Trin Chavalittumrong <tr...@microsoft.com>.
Hi All,
The NTEVentLogAPpender.dll that comes with log4j seems to be working with Windows 32-bit only. Has anyone had any solutions for this problem in Windows 64-bit?
Thanks,
Trin
Re: [compress] add a memory efficient stream compress InputStream
- e.g. a "DeflaterInputStream"
Posted by Patrick Herber <pa...@gmail.com>.
Hello!
Browsing the Internet I found an interesting solution approach to this
problem using piped input/output streams:
InputStream is = ...;
PipedOutputStream pos = new PipedOutputStream();
PipedInputStream pis = new PipedInputStream(pos);
new DataMover(is, new GZIPOutputStream(pos));
stmt.setBinaryStream(2, pis);
stmt.executeUpdate();
private static class DataMover extends Thread {
InputStream in = null;
OutputStream out = null;
public DataMover(InputStream in, OutputStream out) {
super("DataMover");
this.in = in;
this.out = out;
start();
}
public void run() {
try {
byte buf[] = new byte[8192];
int got = -1;
while ((got = in.read(buf)) > 0) {
out.write(buf, 0, got);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
in.close();
} catch (Exception e) {
}
try {
out.close();
} catch (Exception e) {
}
}
}
}
Regards
Patrick
Patrick Herber wrote:
> Hi!
> Thanks a lot for your reply. Please correct me: in practice do you
> suggest changing in my piece of code the line
>
> GZIPOutputStream gos = new GZIPOutputStream(new
> FileOutputStream(tempFile));
>
> with
>
> GzipCompressorOutputStream gos = new GzipCompressorOutputStream(new
> FileOutputStream(tempFile));isn't it?
>
> In case, I imagined that the solution of the issue COMPRESS-48 was
> more something that skipped this whole InputStream to OutputStream to
> InputStream process.
> Something like
>
> InputStream is = ... // original inputStream
> stmt.setBinaryStream(3, new GZIPCompressorInputStream(is), -1);
> // ... "GZIPCompressorInputStream" would be a decorator which compress
> the underlying inputstream
> stmt.executeUpdate();
>
> Is something not feasible?
>
> Thanks again and best regards,
> Patrick
>
>
> Christian Grobmeier wrote:
>> Hi there,
>>
>> I think this one has been addressed in
>> https://issues.apache.org/jira/browse/COMPRESS-83
>> This change is not released yet. You might want to check the code out
>> from SVN and build it yourself.
>> I have some time left later this year and hopefully this change will
>> come with an 1.1 version soon. But this needs to be discussed on the
>> dev list.
>>
>> Let me know if you need help or if this works out for you
>>
>> Best regards,
>> Christian
>>
>> On Tue, Sep 1, 2009 at 12:33 PM, Patrick Herber<pa...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> I've the same issue as described in
>>>
>>> https://issues.apache.org/jira/browse/COMPRESS-48
>>>
>>> My web application recieves an InputStream (an uploaded large file) which I
>>> need to save compressed in the DB.
>>> Currently (since the file could be really big) I save it GZIP-ed into a Temp
>>> File and then I open a new InputStream to that file. However this process
>>> take up more than 60% of the overall method execution:
>>>
>>> tempFile = File.createTempFile(Long.toString(getId()), null);
>>> tempFile.deleteOnExit();
>>> GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream(tempFile));
>>> IOUtils.copy(is, gos);
>>> is = new FileInputStream(tempFile);
>>> stmt.setBinaryStream(3, is, -1);
>>> stmt.executeUpdate();
>>>
>>> The issue is marked as fixed but I didn't find how/where.
>>> Could you kindly give me a sample how to do it?
>>>
>>> Thanks a lot for your help
>>>
>>> Best regards
>>>
>>> Patrick
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>>
>>
Re: [compress] add a memory efficient stream compress InputStream
- e.g. a "DeflaterInputStream"
Posted by Patrick Herber <pa...@gmail.com>.
Hi!
Thanks a lot for your reply. Please correct me: in practice do you
suggest changing in my piece of code the line
GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream(tempFile));
with
GzipCompressorOutputStream gos = new GzipCompressorOutputStream(new
FileOutputStream(tempFile));isn't it?
In case, I imagined that the solution of the issue COMPRESS-48 was more
something that skipped this whole InputStream to OutputStream to
InputStream process.
Something like
InputStream is = ... // original inputStream
stmt.setBinaryStream(3, new GZIPCompressorInputStream(is), -1);
// ... "GZIPCompressorInputStream" would be a decorator which compress
the underlying inputstream
stmt.executeUpdate();
Is something not feasible?
Thanks again and best regards,
Patrick
Christian Grobmeier wrote:
> Hi there,
>
> I think this one has been addressed in
> https://issues.apache.org/jira/browse/COMPRESS-83
> This change is not released yet. You might want to check the code out
> from SVN and build it yourself.
> I have some time left later this year and hopefully this change will
> come with an 1.1 version soon. But this needs to be discussed on the
> dev list.
>
> Let me know if you need help or if this works out for you
>
> Best regards,
> Christian
>
> On Tue, Sep 1, 2009 at 12:33 PM, Patrick Herber<pa...@gmail.com> wrote:
>
>> Hello!
>>
>> I've the same issue as described in
>>
>> https://issues.apache.org/jira/browse/COMPRESS-48
>>
>> My web application recieves an InputStream (an uploaded large file) which I
>> need to save compressed in the DB.
>> Currently (since the file could be really big) I save it GZIP-ed into a Temp
>> File and then I open a new InputStream to that file. However this process
>> take up more than 60% of the overall method execution:
>>
>> tempFile = File.createTempFile(Long.toString(getId()), null);
>> tempFile.deleteOnExit();
>> GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream(tempFile));
>> IOUtils.copy(is, gos);
>> is = new FileInputStream(tempFile);
>> stmt.setBinaryStream(3, is, -1);
>> stmt.executeUpdate();
>>
>> The issue is marked as fixed but I didn't find how/where.
>> Could you kindly give me a sample how to do it?
>>
>> Thanks a lot for your help
>>
>> Best regards
>>
>> Patrick
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
>
Re: [compress] add a memory efficient stream compress InputStream -
e.g. a "DeflaterInputStream"
Posted by Christian Grobmeier <gr...@gmail.com>.
Hi there,
I think this one has been addressed in
https://issues.apache.org/jira/browse/COMPRESS-83
This change is not released yet. You might want to check the code out
from SVN and build it yourself.
I have some time left later this year and hopefully this change will
come with an 1.1 version soon. But this needs to be discussed on the
dev list.
Let me know if you need help or if this works out for you
Best regards,
Christian
On Tue, Sep 1, 2009 at 12:33 PM, Patrick Herber<pa...@gmail.com> wrote:
> Hello!
>
> I've the same issue as described in
>
> https://issues.apache.org/jira/browse/COMPRESS-48
>
> My web application recieves an InputStream (an uploaded large file) which I
> need to save compressed in the DB.
> Currently (since the file could be really big) I save it GZIP-ed into a Temp
> File and then I open a new InputStream to that file. However this process
> take up more than 60% of the overall method execution:
>
> tempFile = File.createTempFile(Long.toString(getId()), null);
> tempFile.deleteOnExit();
> GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream(tempFile));
> IOUtils.copy(is, gos);
> is = new FileInputStream(tempFile);
> stmt.setBinaryStream(3, is, -1);
> stmt.executeUpdate();
>
> The issue is marked as fixed but I didn't find how/where.
> Could you kindly give me a sample how to do it?
>
> Thanks a lot for your help
>
> Best regards
>
> Patrick
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org