You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Hayward, Leigh" <lh...@student.le.ac.uk> on 2014/04/30 16:31:33 UTC
Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of the
program instead of the current run.
Hello all,
My Java EE web application takes in multiple audio inputs and outputs
them as a single wav file via an application/octet stream.
It seemingly randomly works correctly (i.e. outputting the correctly
manipulated audio file) but sometimes, the file from a previous
run of the program is output and I get one of these errors:
SEVERE [http-nio-8084-exec-30] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/MyApp] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
and to me seemingly random numbers of these errors:
"SEVERE [http-nio-8084-exec-87] org.apache.coyote.http11.AbstractHttp11Processor.process Error processing request java.lang.IllegalStateException: The resources may not be accessed if they are not currently started?"
The files always upload correctly to my filesystem, but something is
going wrong when I try to access them in order to process them.
Also when it is downloading the file it appears to the user
to be several MB long despite the file that is output being only a few
thousand KB.
Sometimes when it doesn't work it does not produce these errors, but it
never produces these errors when it works correctly.
I have googled it but there's no reference to the second kind of error
anywhere on the web aside from svn commits by tomcat developers,
so while I am a total newbie to mailing lists, after exhausting
stackoverflow this seemed like the only place to turn to. I'm
developing a java EE web application in Netbeans using Tomcat
8.0.3.0 on a windows 7 operating system.
The web application is very basic and allows users to upload files via a
multipart html form. This then posts to a servlet which first uploads these
files to the programs file system and then accesses them, concatenates
them together and saves them back on the file system.
It's better explained by this diagram http://imgur.com/Oacd4gq
Could it be that the files are not being uploaded fully before they are
being accessed?
Sorry if this email looks like shit i'm being forced to use outlook
by my university!
Thank you for taking the time to read my post.
Best wishes to you all regardless,
Leigh
Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of
the program instead of the current run.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Leigh,
On 5/1/14, 9:22 AM, L. HAYWARD wrote:
> https://github.com/Leighbee13/RUNTHIS/tree/master
For your first web application, you just *had* to use jQuery, eh?
Okay, I've got no jQuery experience, but it looks like when you click
the "Add Song" button, the add_more handler runs:
$('#add_more').click(function() {
var current_count;
if ($('.addsort').length !== 0) {
current_count =
parseInt($('#inputs').children(':last-child').children().first().attr('name'));
}else{
current_count=1;
}
var next_count = current_count + 1;
$('#sortable').append('<li id= "song' + next_count + '"
name="'+next_count+
'" class="ui-state-default"><span class="ui-icon
ui-icon-arrowthick-2-n-s"></span> <img class="deletesong"
src="images/delete.png" alt="del" width="20" height="20" style="float:
right;"></li>');
$('#inputs').append('<p><input type="file" id="file_' +
next_count + '" class = "addsort" name ="' + next_count + '"
accept=".wav, .au*" required/></p>');
});
This is the markup from your JSP:
<div id="inputs">
<p><button type='button' id="add_more"
style="float: right;">Add Song</button><!--<a id="add_more"
href="#">Add song</a>--></p>
<h3>Songs:</h3>
<p> <input type="file" class= "addsort"
id="file_1" name ="1" accept=".wav, .au*" required/></p>
</div>
If I read the jQuery code correctly, it's going to choose the first
child of the last child of the DIV and use that element's "name" as
the current_count, then add one to it to get the next_count.
I believe the first child of the last child of the DIV is actually the
<button>, which would make the current_count nil, as it has no "name"
attribute. If I've got this wrong, please correct me.
You might want to eliminate the jQuery from the equation and just
hard-code the <form> to have some specific number of file inputs and
just leave some of them blank for testing. You can always check for
null-ness on the server side (and you should, since the user could
always leave one of the dynamically-added ones blank).
Okay, comments on the actual Java code:
1. Do not have doGet call doPost. It's not going to work. Just remove
goGet entirely, and the parent class will return a "Method Not
Supported" error to the client.
2. What does the stdout log usually look like for a "good" run? How
about a "bad" one?
3. If the part.write() call fails, you are screwed. You shouldn't just
(log and) ignore the exception. Instead, you should blow up and tell
the user that something bad happened.
4. What's the "program select" thing? Does that interact with the
other audio clips uploaded? Why are you using a java.util.Scanner?
5. When you call requestDispatcher.forward(), you need to return from
the doPost method ASAP. The way your servlet is written, the rest of
the servlet will continue on running (trying to overlay the songs,
etc.) and chaos will ensue. You can simply put a "return;" after the
call to requestDispatcher.forward(). I'm specifically looking at the
forward() call at line 109.
6. I'd recommend that you use an array of java.io.File objects as the
parameter to your overlay() method, but that's more of a matter of
taste than anything that will affect your program. You could also use
a set of InputStreams and writing the individual audio files to the
disk manually. I'm not sure that would improve anything other than a
bit of a performance gain. It would simplify your code, which is
always a nice thing to do. Feel free to ignore this entire comment, as
it shouldn't change anything about the correctness of your code.
7. You are allocating enormous byte buffers to read-in the audio clips
in their entirety. Is that strictly necessary? Also, you are
truncating long values to int values. For your testing, I assume that
you are never exceeding Integer.MAX_VALUE, but you could be surprised
by something like that. I realize this is "toy code" but if every
dissertation would include decent code, the world would be a better place.
8. You are doing a lot of manipulations across many buffers that could
probably be combined into fewer operations: reading bytes, packing
bytes, summing integers, unpacking bytes, then re-encoding as an audio
clip.
9. Your overlay() method should probably take the name of the target
file as a parameter. That way, the caller will know where the file is
going. Then, provide the same file name to the finish() method. That
way, the target file name is only specified in a single place, and you
can change it whenever you want (like if you want to make this thing
work with multiple users, it needs no further modification if the
doPost method chooses its own unique target file name for each request).
10. In your finish() method, you should buffer the reads and writes to
improve performance. Feel free to use a BufferedInputStream to read
the file, or you could just use your own byte[] buffer. Processing
files one byte at a time sucks. You might think that FileInputStream
and ServletOutputStream would be buffered, but you'd be wrong.
11. In concatenate, you start with songs/noth.wav (an empty WAV file?)
and then you concatenate each file, one at a time, each time, so you
end up with a bunch of files named test0.wav, test1.wav, test2.wav,
etc. each successive one being the contents of the previous one plus
one additional song, right? I think you can shorten the whole thing by
opening all the files at once as AudioInputStreams, then packing them
into an Enumeration, and passing-off the whole thing to
AudioSystem.write, like this:
public void concatenate(String[] songs, String targetFilename)
throws Exception
{
long totalFrameCount = 0;
ArrayList<InputStream> inputs = new ArrayList<>(1 + songs.length);
AudioInputStream nothing = loadToWav("sound/noth.wav");
inputs.add(nothing);
totalFrameCount += nothing.getFrameLength();
AudioFormat fmt = nothing.getFormat();
for(String input : songs) {
AudioInputStream ais = loadToWav(input);
inputs.add(ais);
totalFrameCount += ais.getFrameLength();
}
SequenceInputStream sis = new
SequenceInputStream(Collections.enumeration(inputs));
AudioInputStream in = new AudioInputStream(sis, fmt, totalFrameCount);
AudioSystem.write(in,
AudioFileFormat.Type.WAVE,
new File(targetFilename));
in.close();
sis.close(); // Maybe unnecessary?
}
I have a dumb question: if WAV uses frames, can't you just concatenate
all of the bytes together from the source files and not worry about
AudioInputStreams and stuff like that? I did a bit of Googling and
nobody seems to have said that simply concatenating the bytes of a
bunch of WAV files together will actually work. Is the problem that
you have to make sure they're all of the same type, and then you'd
have to remove the headers as you perform the concatenation? I suppose
it's just easier and less fragile to formally decompress the audio and
then re-encode it. It just seems like kind of a waste.
I do know that you can simply concatenate MP3 files in the way I've
described, because there is no file header. But if you aren't careful,
you'll get noisy pops and stuff when two adjacent frames (at the file
boundary) introduce a discontinuity in the audio.
There's a lot of stuff in there that I don't understand, but I'm
hoping that with a few tweaks, you can get your problem solved.
I'm fairly confident that #5 above is going to fix 90% of the problem
you have been having.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTapr2AAoJEBzwKT+lPKRYe6YP/R202RoQtVXdsON407Lw+LXm
L90DlwfSmUpVTAdQu8f07f0T/0FCcB5vuDpS/JfHNdwC7F/p0S90iMyrgdI1oM5S
MHeC0Y+uF/z4cPme7lNvcQA/E1zw9+GC65vUWa0BELW42I563b1IyxEG+Ec+i2nS
a4Zl9ub3gtx3FK1SYtDr93pnIwbFEJgPHynNQxbQcBlbeclaLIwLdDYd7UUIkCyc
tJkIhtXsmOYGp/bmG3lZHrgUj7RjR9AztEl61jWxItFpg7oWC6gJI34CTOCNqCjl
r2qOtLbBSWVnwBtNoKs2e5Ve01tIWs9sWamZhre8GpitDv3kj+ODu7qe/a5TWJup
YNs/VgVr/w62P/UVhpo5Je6ml3AqRXrWt40QH+kxcIPeOc6Y7E3ELIbtUGLzLnQW
+b2BwSUfi9yFaYjxvmKVbri7vCvs0I857F8pY/BDN3t+s7+nkgRfB+JTQwmqb22/
SlVTQ4JJfnkurUyu/iu/BQB2KcdjFPuUTtLXSO0/OHnEGq1RZZy5AiLE2U9Wyfxm
bHvTHsCMmelcLVdRSAi2ul7HVUvTZgZPVgGO7lPzAdy4qSN8OK+g8yDjeQAS3Jz3
Zj/+Gh3mhD2Bgf+Bc9CTE/2YKtasZ5tbWvW/1HiQ/uwyMT2uMPbLvSfZvoIbb3ba
Mo0XovV3o0v/IR2JZ9u+
=7Tww
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of
the program instead of the current run.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Leigh,
On 5/1/14, 9:22 AM, L. HAYWARD wrote:
> On 30/04/14 23:39, Konstantin Kolinko wrote:
>> 2014-05-01 2:00 GMT+04:00 Hayward, Leigh
>> <lh...@student.le.ac.uk>:
>>> OK so I just spent an age trying to push my project to a
>>> GitHub repositry but I think i must just be too tired to use my
>>> brain correctly. Eventally gave up and just pasted my servlet
>>> into a file there.
>>> https://github.com/Leighbee13/RUNTHIS/tree/master Let me know
>>> if this works!
>>
>> An interesting servlet...
>
> Yeah this is my first thing that I've made myself alone from start
> to finish so it's not good.
I haven't looked at the code, yet, but don't sell yourself short. You
are trying to do some interesting work (the audio stuff) and then slap
some plumbing around it to get it to work. This is uncharted territory
for you. It may be a mess, but having other people look at your code
is a good way to improve.
If I knew I could get a PhD by writing a servlet, I'd have done it a
long time ago :)
>> The problem is that the names of your temporary files are
>> constant ones.
>>
>> So if several requests are coming at the same time. several
>> users' data will be written into the same file.
>>
>
> I am just writing this for a dissertation project so it's just
> handling 1 request right now, from me, in netbeans. So this error
> shouldn't be to do with multiple requests, right?
>
> I did a test and used a random number generator to generate file
> names instead of a count and it gives me null pointer errors when
> trying to concatenate now.
>
> When it was overwriting, if it couldn't find the file it needed it
> would just use the previously successfully uploaded file instead
> of nullpointer-ing. But why isn't the file correctly uploaded by
> the time the concatenate method try's to access it?
This might have to do with the error you are getting. See below.
> What does "java.lang.IllegalStateException: The resources may not
> be accessed if they are not currently started?" mean?
It sounds like your web application is restarting itself -- probably
due to modifications to the code. When Tomcat is configured to
auto-reload applications, you can trigger a re-load by re-compiling a
class. In something like NetBeans, that's probably trivial do to...
just save the file and it will probably re-compile and re-deploy.
Try not doing that when running your tests... it may solve a great
deal of your problems.
>> If a user hasn't completed downloading her file, and another
>> requests comes in, the "finished.wav" file will be overwritten
>> with new data.
>>
>> Best regards, Konstantin Kolinko
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
> I screwed up the formatting for this email since I was being forced
> to use outlook at uni, but now I'm on a linux machine with
> thunderbird so it shouldn't happen again. The original question had
> a better explanation and the errors that I was getting.
>
> It seemingly randomly works correctly (i.e. outputting the
> correctly manipulated audio file) but sometimes, the file from a
> previous run of the program is output instead, and I get one of
> these errors:
>
> SEVERE [http-nio-8084-exec-30]
> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
> The web application [/MyApp] is still processing a request that has
> yet to finish. This is very likely to create a memory leak. You can
> control the time allowed for requests to finish by using the
> unloadDelay attribute of the standard Context implementation.
You are definitely re-loading your web application. When this happens,
a new copy of the webapp is deployed and can handle requests with your
new code. If the old code is still actively running (i.e. still
processing a request), you can get .. confusing results.
You should take a thread dump [1] when this happens. It will tell you
what code is (still) executing. Make sure you do it before you make
another request to your web application.
> (n.b. i did use unloadDelay and it made no difference.)
>
> and to me seemingly random numbers of these errors:
>
> "SEVERE [http-nio-8084-exec-87]
> org.apache.coyote.http11.AbstractHttp11Processor.process Error
> processing request java.lang.IllegalStateException: The resources
> may not be accessed if they are not currently started?"
The web application has officially stopped, but some code is still
trying to operate within it. If you have a long-running request that
survives the re-start (as evidenced above), then you'll get weird
errors like this.
> These second errors are ungooglable. The only reference to them is
> on svn commits from tomcat devs.
>
> The files always upload correctly to my filesystem, but something
> is going wrong when I try to access them in order to concatenate
> them.
>
> It sometimes produces the wrong output but with no errors, but the
> errors never occur when the expected output is created.
Good to know.
> I hope that makes sense. I've added my jsp, javascript and css to
> my github repositry to see if that helps!
>
> https://github.com/Leighbee13/RUNTHIS/tree/master
I'll take a look at both sets of files. Hopefully it's not too much
code. Remember to thank the Tomcat community in your thesis
publication. ;)
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTaoqzAAoJEBzwKT+lPKRYzJsQAMXWX5jJ+PUxlSagSa3bZkFQ
LaF8NYxacftGghKWW76jhYaPYp49lu4dW+kyshJI4KNGWbZXc8uD8L+u3QeyklJ1
FdcfaX3+M/ZZdcEsfl61BCDTCf0QBcdDLjM2T/agAklOu2sxPWf/R9QoGf/Icnzk
yJHwILsF3vyhfx0UysNm4rq1/ZLiSqOO2XsbCWUky0zClGE5IzbqRSWVGTn2gDRT
nTSWXEyEBPozyInGqbyZlJk460I9OD78l3OXdmxP/67fQx2rNUfC/tyKVHxU4TS5
lIw9F+cJyuZKXxT37MD75w3p8j0g135bhGTCHXvnLchoF3vExZAXi10oIkPHJWIy
0Po+oRgqGsLdW9iFomhh8DsPDIX1bgZRUlRgi78mE6Z61bKt0fJUKa8Ug1QqoQTH
IB16fOAhCCD3wwwAhPrcCfQnQrRgdxaaQiY6r2fQkHlDQ8WC9eQedBtECK+w5zdl
+eoK9qOKagsvfjQZ3N8qrusxfL7vx1hSLPoESkxvFXF2AqhCIKyvz4ut1RBckA0i
eviUXNXpiOJW099D+L3duw5X9QnYjPAIbm9eCZp2U2sfmaYVXyzNxzZjCdMt0hNJ
kXXy9TisVlfKl8MR9bUIewalAq+xWL1vSoV3VgrwNaTbWScc2p68schHxrDpYMTT
rjGN0Nca+fZJ5kr6ynmm
=z2Ts
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of
the program instead of the current run.
Posted by "L. HAYWARD" <lh...@student.le.ac.uk>.
On 30/04/14 23:39, Konstantin Kolinko wrote:
> 2014-05-01 2:00 GMT+04:00 Hayward, Leigh <lh...@student.le.ac.uk>:
>> OK so I just spent an age trying to push my project to a GitHub repositry but
>> I think i must just be too tired to use my brain correctly.
>> Eventally gave up and just pasted my servlet into a file there.
>> https://github.com/Leighbee13/RUNTHIS/tree/master
>> Let me know if this works!
>
> An interesting servlet...
Yeah this is my first thing that I've made myself alone from start to
finish so it's not good.
>
> The problem is that the names of your temporary files are constant ones.
>
> So if several requests are coming at the same time. several users'
> data will be written into the same file.
>
I am just writing this for a dissertation project so it's just handling
1 request right now, from me, in netbeans. So this error shouldn't be to
do with multiple requests, right?
I did a test and used a random number generator to generate file names
instead of a count and it gives me null pointer errors when trying to
concatenate now.
When it was overwriting, if it couldn't find the file it needed it would
just use the previously successfully uploaded file instead of
nullpointer-ing. But why isn't the file correctly uploaded by the time
the concatenate method try's to access it?
What does "java.lang.IllegalStateException: The resources may not be
accessed if they are not currently started?" mean?
> If a user hasn't completed downloading her file, and another requests
> comes in, the "finished.wav" file will be overwritten with new data.
>
> Best regards,
> Konstantin Kolinko
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
I screwed up the formatting for this email since I was being forced to
use outlook at uni, but now I'm on a linux machine with thunderbird so
it shouldn't happen again. The original question had a better
explanation and the errors that I was getting.
It seemingly randomly works correctly (i.e. outputting the correctly
manipulated audio file) but sometimes, the file from a previous
run of the program is output instead, and I get one of these errors:
SEVERE [http-nio-8084-exec-30]
org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The
web application [/MyApp] is still processing a request that has yet to
finish. This is very likely to create a memory leak. You can control the
time allowed for requests to finish by using the unloadDelay attribute
of the standard Context implementation.
(n.b. i did use unloadDelay and it made no difference.)
and to me seemingly random numbers of these errors:
"SEVERE [http-nio-8084-exec-87]
org.apache.coyote.http11.AbstractHttp11Processor.process Error
processing request java.lang.IllegalStateException: The resources may
not be accessed if they are not currently started?"
These second errors are ungooglable. The only reference to them is on
svn commits from tomcat devs.
The files always upload correctly to my filesystem, but something is
going wrong when I try to access them in order to concatenate them.
It sometimes produces the wrong output but with no errors, but the
errors never occur when the expected output is created.
I hope that makes sense. I've added my jsp, javascript and css to my
github repositry to see if that helps!
https://github.com/Leighbee13/RUNTHIS/tree/master
Thanks so much for taking a look,
Leigh
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of the
program instead of the current run.
Posted by Konstantin Kolinko <kn...@gmail.com>.
2014-05-01 2:00 GMT+04:00 Hayward, Leigh <lh...@student.le.ac.uk>:
> OK so I just spent an age trying to push my project to a GitHub repositry but
> I think i must just be too tired to use my brain correctly.
> Eventally gave up and just pasted my servlet into a file there.
> https://github.com/Leighbee13/RUNTHIS/tree/master
> Let me know if this works!
An interesting servlet...
The problem is that the names of your temporary files are constant ones.
So if several requests are coming at the same time. several users'
data will be written into the same file.
If a user hasn't completed downloading her file, and another requests
comes in, the "finished.wav" file will be overwritten with new data.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8.0.3.0 getting never before seen by google Illegal
State Exception. Sevlets outputting the audio output from the previous runs
of the program instead of the current run.
Posted by "Hayward, Leigh" <lh...@student.le.ac.uk>.
OK so I just spent an age trying to push my project to a GitHub repositry but
I think i must just be too tired to use my brain correctly.
Eventally gave up and just pasted my servlet into a file there.
https://github.com/Leighbee13/RUNTHIS/tree/master
Let me know if this works!
>________________________________________
>From: Christopher Schultz <ch...@christopherschultz.net>
>Sent: 30 April 2014 20:13
>To: Tomcat Users List
>Subject: Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State Exception. Sevlets outputting the >audio output from the previous runs of the program instead of the current run.
>-----BEGIN PGP SIGNED MESSAGE-----
>>Hash: SHA256
>Leigh,
>On 4/30/14, 1:55 PM, Hayward, Leigh wrote:
>> Hi Chris, Thank you so much for the speedy response . I have no
>> idea what i'm doing formatting wise. Pretty much just wrapping and
>> indenting all by hand so forgive me if it's a mess!
>>
>>> ________________________________________ From: Christopher
>>> Schultz <ch...@christopherschultz.net> Sent: 30 April 2014 17:58
>>> To: Tomcat Users List Subject: Re: Tomcat 8.0.3.0 getting never
>>> before seen by google Illegal State Exception. Sevlets outputting
>>> the >audio output from the previous runs of the program instead
>>> of the current run.
>>
>> Leigh,
>>
>> On 4/30/14, 10:31 AM, Hayward, Leigh wrote:
>>>> My Java EE web application takes in multiple audio inputs
>>>> and outputs them as a single wav file via an
>>>> application/octet stream.
>
> Like a mixer?
>
>> Like a concatenator, adds songs one after the other into one long
>> song. It then overlays (mixes) another file into it containing
>> audio cues on top of the concatenated music. But the issue is
>> definitely arising at the concatenation phase.
>Okay. Are you saying that you don't get the output file you are
>expecting given a set of input files, even when under test conditions
>(a single request)?
>Do you accept all files at once for upload and return a single
>concatenated file, or do you upload the files one at a time and then
>request the completed file at the end of the workflow?
Yes i'm not getting the output I expect given the inputs, i'm getting
the output that would have happened with a previous input.
Yes the form has an unknown number of file inputs and they are
all submitted at once.
>> I have the audio writing to a file after each song is added in
>> the concatenation phase so that i can audibly debug and the
>> concatenator is definitely struggling to find the uploaded
>> files.
>What do you mean "struggling to find the uploaded files"? Your file
>upload library should be readily giving the files to you.
By struggling I just mean that the concatenate method is where the
incorrect audio output begins being produced instead of the correct
one. At least as far as i can tell from my
>>>> It seemingly randomly works correctly (i.e. outputting the
>>>> correctly manipulated audio file) but sometimes, the file
>>>> from a previous run of the program is output and I get one of
>>>> these errors:
>>>>
>>>> SEVERE [http-nio-8084-exec-30]
>>>> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
>>>>
>>>>
The web application [/MyApp] is still processing a request that has
>>>> yet to finish. This is very likely to create a memory leak.
>>>> You can control the time allowed for requests to finish by
>>>> using the unloadDelay attribute of the standard Context
>>>> implementation.
>>>>
>>>> and to me seemingly random numbers of these errors:
>>>>
>>>> "SEVERE [http-nio-8084-exec-87]
>>>> org.apache.coyote.http11.AbstractHttp11Processor.process
>>>> Error processing request java.lang.IllegalStateException: The
>>>> resources may not be accessed if they are not currently
>>>> started?"
>
>
> These kinds of things are almost always due to storing of a request
> or response option in some kind of structure that survives past the
> end of a particular request.
>
> Can you explain how you build the response -- specifically
> involving any non-standard threading you may do?
>
>> The response is built in a method called finish which is called
>> in doPost like this :
>
>> finish(req, res); In finish I make a printwriter I set response
>> content type to "application/octet-stream", set the header
>> "Content-Disposition" to "attachment; filename=finished.wav" I
>> make a FileInputStream of the finished filepath Then I set the
>> header content length to the length of the finished file. then I
>> while loop the fileInputStream into the printwriter Then I close
>> the FIS.
>That seems fairly straightforward. Why are you bothering with a
>PrintWriter? The existing response OutputStream ought to be good
>enough to pump bytes through. How are you pumping the bytes? Just a
>simple, locally-declared byte array?
The online examples use printwriters so I used a printwriter.
I've now changed it to the res's own output stream but it made no
difference to the output.
I'm just using:
while ((i = fileInputStream.read()) != -1) {
out.write(i);
}
fileInputStream.close();
out.close();
>You should remember to flush and/or close the output stream as well. I
>think Tomcat will flush and close this for you, but you might want to
>check to see if it changes anything
.
Sorry i did close the printwriter as well just forgot to mention it!
I'm now flushing it too. but it makes no difference.
>> I have not touched threading since it looks terrifying. I did
>> experiment with making the servlet singlethreaded since I thought
>> it could have something to do with threads (bad practice I know)
>> but that did nothing to help.
>Are you using any variables that have been declared at the class
>level? Everything you do with a request should be declared inside of
>the service method (or doPost, etc.) or a parameter passed-into a
>utility method. Don't store references to request, response, etc.
>outside of the thread of execution that is handling that request.
>If single-threading fixes your problem (sounds like it doesn't), then
>you have broken the above rule.
Yeah I read this article
http://www.javaworld.com/article/2072798/java-web-development/write-thread-safe-servlets.html
And removed all of my variable declared outside their respective
classes and it didn't help, then I single-threaded it and that didn't
work either. Does that rule out threads as being the potential
problem?
>>>> The files always upload correctly to my filesystem, but
>>>> something is going wrong when I try to access them in order
>>>> to process them.
>
> What mechanism do you use for upload?
>
>> My servlet has a location set in the @multipartconfig tag that is
>> where all files are saved to and retrieved from. I am getting the
>> Parts from the request then iterating through them, for each part
>> that is a file upload I use part.write("uploadedfile" + count +
>> ".wav");
>In that case, the files should all have arrived on the disk before
>your code gets a chance to read them. Remember that they aren't
>required to be on the disk: you should be reading them from the
>InputStream(s) that the container (Tomcat) gives you. Also remember
>that the container (Tomcat) will probably delete those files after the
>request has completed, so you can't rely on them being in any
>particular place for any amount of time: if you want a copy of a file,
>you'll have to make it yourself.
What do you mean by not required to be on the disk? As in I should be
copying files into that file again? Nothing appears to be being deleted
after I run it. When I go to the sound file, all the files are still there
after the run is over.
>>>> Also when it is downloading the file it appears to the user
>>>> to be several MB long despite the file that is output being
>>>> only a few thousand KB.
>
> NB a few thousand KiB is the definition of several MiB.
>
>> I am possibly just an idiot on that one. I think the deadline
>> sleep deprivation is getting to me
>It happens to all of us.
Glad to hear it!
> Is the response built before you stream any of it to the client?
> Are you setting a Content-Length before you send any data? Are you
> using chunked responses?
>
>> I call a finish(req, res) method in my doPost which is where all
>> my response building is done. I am setting a content length
>> before I send the data because I wanted to be able to see how
>> long it was going to take to download since the program is fairly
>> slow.
>It doesn't seem like concatenating files would take that much time,
>besides the upload time. I don't know much about WAV files, though...
>perhaps you have to do the math to connect them together properly. If
>you have to re-encode entire files, though, you are probably doing it
>wrong.
Oh I am most certainly doing something wrong! I'm not too concerned
with speed right now i just want it to output the right thing.
To connect them I'm just using a SequenceInputStream.
See the GitHub.
>>>> Sometimes when it doesn't work it does not produce these
>>>> errors, but it never produces these errors when it works
>>>> correctly.
>
> Does (increased) load seem to make the situation worse?
>
>> Definitely. Always happens with long audio files and only
>> sometimes with shorter test ones.
>So long files end up breaking things more than short files? When I
>said "load", I really meant "numbers of simultaneous requests"
>(increasing).
There will never be more than one user as it will never really go
online.
>If you can't even handle one isolated request with long files, you
>have a serious problem.
Tell me about it. Honestly as long as I eventually get the output
I'm looking for I don't mind.
>>>> I have googled it but there's no reference to the second kind
>>>> of error anywhere on the web aside from svn commits by
>>>> tomcat developers, so while I am a total newbie to mailing
>>>> lists, after exhausting stackoverflow this seemed like the
>>>> only place to turn to. I'm developing a java EE web
>>>> application in Netbeans using Tomcat 8.0.3.0 on a windows 7
>>>> operating system.
>
> (You should upgrade to a later Tomcat 8 beta. Nothing should
> affect what you are seeing here, but it's not a bad idea to get the
> latest.)
>
>> I'm just a little nervous about moving or touching anything since
>> i'm not 100% confident it wont just break everything. Still new
>> at this. I wanted the most stable version, to balance out my
>> intellectual instability.
>The most stable version of 7.0.53. The 8.0.x line hasn't yet been
>voted stable, though it pretty much is. Latest 8.0.x is 8.0.5 and will
>soon be 8.0.6, but it has only been voted beta at this point.
Do I have to? I know you're the expert and I trust that what you're
saying is completely correct and all I just really don't want to mess
with things too much this late in the game.
>>>> The web application is very basic and allows users to upload
>>>> files via a multipart html form. This then posts to a servlet
>>>> which first uploads these files to the programs file system
>>>> and then accesses them, concatenates them together and saves
>>>> them back on the file system. It's better explained by this
>>>> diagram http://imgur.com/Oacd4gq
>
> The real question is whether you are using non-request-processor
> threads to do any of this work. Also, where are the various data
> stored while you are working on them? Not the filesystem -- that
> seems self-evident -- but where in the program are the String[]
> objects that represent all the songs that have been uploaded?
>
>> C:\Users\Leigh\Documents\NetBeansProjects\MyApp\src\java\sound
>
>> All audio files get stored in and are accessed from there.
>No, somewhere in your program you have a String[] declared where you
>have a reference to those filenames. You shouldn't actually have
>references to path names at all, because the container can buffer the
>files in memory instead of ever writing them to the disk.
Ah sorry, the string [] songs gets created in doPost by having a counter
in my parts iterator that counts how many audio files there are and
then a string [] is created of the song file locations them using a for
loop. It is then put into the overlay method as a variable.
>Anyhow, I was wondering if you were using a String[] declared at the
>servlet level to track this stuff. That always results in disaster,
>because multiple requests will fight over the same variable and step
>on each other's toes.
>Any chance you could post your code somewhere for inspection? Dropping
>it right into a post to the list is just fine.
Done :)
>> This is purely a dissertation project so I have no intention of
>> making it work for multiple users or in fact be functional
>> anywhere outside of my laptop. All files are being uploaded to
>> the file system of the project itself since that was the only way
>> I knew how to make it work.
>Why bother with a web application interface, then?
Because this would, in the world that is the context of my
dissertation, be a web application. I don't need it to be
one in the real world. I just need to display that can use
technologies like CSS, HTML and Servlets in order to get
a good grade.
> When you process a "transaction" (upload -> merge -> respond), are
> you taking care to ensure that the files are placed into different
> paths for each request? How are you choosing the filename of the
> merged audio clip?
>
>> all uploads are called uploadedfile1, uploadedfile2,
>> uploadedfile3 etc depending on how many files are uploaded.
> If these files all go to the same place, then multiple requests will
> overwrite each other. That's going to be a problem, except you said
> you don't care. Honestly, you should either care about this and make
> it work properly, or just drop the "web" requirement where it's okay
> to be sloppy.
It's not that I don't care and i can't drop the web requirement as it's a
university project. Again, there ill never be multiple requests it's a
purely theoretical project. All I want is for it to output the files that i
am inputting concatenated and overlaid.
>> When these files don't exist already on my file system I
>> sometimes get null pointer errors and an empty output.
>> Overwriting works better than creating new.
>
>> The filename of the finished file is just called finished.wav.
>> Always. It just gets overwritten each time I run it. Or not as it
>> seems.
I would choose a different filename every time. Especially if it looks
like the old copy of the file hasn't been closed entirely, yet.
>>>> Could it be that the files are not being uploaded fully
>>>> before they are being accessed?
>
> I don't believe this is possible when using Tomcat's file-upload
> implementation. It is certainly possible to do if you are
> performing the multipart handling yourself. What tool are you using
> to fetch the multipart data from the request?
>
>> As I said above I just get the parts from the request, for loop
>> through them and write any audio files to the locations specified
>> in my MultipartConfig.
Looking at the code would be very helpful. Feel free to leave-out any
cutting-edge research-y stuff that you don't want to divulge ;)
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTYUtlAAoJEBzwKT+lPKRYrtsQAKaQHBjvC8i4UJngL3mDtGMD
t3JxMET3Nb/AGAkeBFn2yJR+PTR4Vuj1gbEKrHN+SE3l1VlbVmjxfOQlPDAm3SPF
SE50zTnmYKKdcOMHpIuUeiMgJh/uKCym+bp3PNcuvORZq/FVJdJ3b8me0MQUO3Vh
juCqj/h/UV+eKZaFmLA+dcJAQoxmHgVoCidnwRN255LUve8/o/JQXthTjmCGuf4Z
rQ7ZarxVr8wbXZJ5wQZ/O1Z6K7IIZ07hNbDHCtj+3ZEeEcSzn5IEOMpskhSu48bC
5LTqn5gXgE7m+uFszjjpshRhoO2PPRf2sb39IYVkkExqVb5EOkdNpVVckRaBd371
H14S/CoWQcCvfq+QORl/i95vrMf+P7mMqD+wS3Bagyw3zD0spTu48oLcyG2Crmwd
zCFyezw3SIJdis8yQBsl7uGNMELF5gJ6n0oPvcUwjiY7tHbZ6f6eF5aEPAz/r1eS
uf3OHeqJswRW9qBUGPxEtvNOajs4q0o6BacfQpiQlHeLNzh6OzGsLmzUZJGoWiPV
zJR+52Z+5RFXDI38E+XSX/T9UFF7M1WjH0mfjpYz6TjIDZl7H+FH7Od5+tQ02M4C
tlr66gq2HqpQA1SlhnRpIpSsyF61PIJIteX1rcRW6qik6qizn4XJQ8GoweCwV8Jn
F2ASYC6UsDVoHpHtJfrg
=uGx4
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of
the program instead of the current run.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Leigh,
On 4/30/14, 1:55 PM, Hayward, Leigh wrote:
> Hi Chris, Thank you so much for the speedy response . I have no
> idea what i'm doing formatting wise. Pretty much just wrapping and
> indenting all by hand so forgive me if it's a mess!
>
>> ________________________________________ From: Christopher
>> Schultz <ch...@christopherschultz.net> Sent: 30 April 2014 17:58
>> To: Tomcat Users List Subject: Re: Tomcat 8.0.3.0 getting never
>> before seen by google Illegal State Exception. Sevlets outputting
>> the >audio output from the previous runs of the program instead
>> of the current run.
>
> Leigh,
>
> On 4/30/14, 10:31 AM, Hayward, Leigh wrote:
>>>> My Java EE web application takes in multiple audio inputs
>>>> and outputs them as a single wav file via an
>>>> application/octet stream.
>
> Like a mixer?
>
>> Like a concatenator, adds songs one after the other into one long
>> song. It then overlays (mixes) another file into it containing
>> audio cues on top of the concatenated music. But the issue is
>> definitely arising at the concatenation phase.
Okay. Are you saying that you don't get the output file you are
expecting given a set of input files, even when under test conditions
(a single request)?
Do you accept all files at once for upload and return a single
concatenated file, or do you upload the files one at a time and then
request the completed file at the end of the workflow?
>> I have the audio writing to a file after each song is added in
>> the concatenation phase so that i can audibly debug and the
>> concatenator is definitely struggling to find the uploaded
>> files.
What do you mean "struggling to find the uploaded files"? Your file
upload library should be readily giving the files to you.
>>>> It seemingly randomly works correctly (i.e. outputting the
>>>> correctly manipulated audio file) but sometimes, the file
>>>> from a previous run of the program is output and I get one of
>>>> these errors:
>>>>
>>>> SEVERE [http-nio-8084-exec-30]
>>>> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
>>>>
>>>>
The web application [/MyApp] is still processing a request that has
>>>> yet to finish. This is very likely to create a memory leak.
>>>> You can control the time allowed for requests to finish by
>>>> using the unloadDelay attribute of the standard Context
>>>> implementation.
>>>>
>>>> and to me seemingly random numbers of these errors:
>>>>
>>>> "SEVERE [http-nio-8084-exec-87]
>>>> org.apache.coyote.http11.AbstractHttp11Processor.process
>>>> Error processing request java.lang.IllegalStateException: The
>>>> resources may not be accessed if they are not currently
>>>> started?"
>
>
> These kinds of things are almost always due to storing of a request
> or response option in some kind of structure that survives past the
> end of a particular request.
>
> Can you explain how you build the response -- specifically
> involving any non-standard threading you may do?
>
>> The response is built in a method called finish which is called
>> in doPost like this :
>
>> finish(req, res); In finish I make a printwriter I set response
>> content type to "application/octet-stream", set the header
>> "Content-Disposition" to "attachment; filename=finished.wav" I
>> make a FileInputStream of the finished filepath Then I set the
>> header content length to the length of the finished file. then I
>> while loop the fileInputStream into the printwriter Then I close
>> the FIS.
That seems fairly straightforward. Why are you bothering with a
PrintWriter? The existing response OutputStream ought to be good
enough to pump bytes through. How are you pumping the bytes? Just a
simple, locally-declared byte array?
You should remember to flush and/or close the output stream as well. I
think Tomcat will flush and close this for you, but you might want to
check to see if it changes anything.
>> I have not touched threading since it looks terrifying. I did
>> experiment with making the servlet singlethreaded since I thought
>> it could have something to do with threads (bad practice I know)
>> but that did nothing to help.
Are you using any variables that have been declared at the class
level? Everything you do with a request should be declared inside of
the service method (or doPost, etc.) or a parameter passed-into a
utility method. Don't store references to request, response, etc.
outside of the thread of execution that is handling that request.
If single-threading fixes your problem (sounds like it doesn't), then
you have broken the above rule.
>>>> The files always upload correctly to my filesystem, but
>>>> something is going wrong when I try to access them in order
>>>> to process them.
>
> What mechanism do you use for upload?
>
>> My servlet has a location set in the @multipartconfig tag that is
>> where all files are saved to and retrieved from. I am getting the
>> Parts from the request then iterating through them, for each part
>> that is a file upload I use part.write("uploadedfile" + count +
>> ".wav");
In that case, the files should all have arrived on the disk before
your code gets a chance to read them. Remember that they aren't
required to be on the disk: you should be reading them from the
InputStream(s) that the container (Tomcat) gives you. Also remember
that the container (Tomcat) will probably delete those files after the
request has completed, so you can't rely on them being in any
particular place for any amount of time: if you want a copy of a file,
you'll have to make it yourself.
>>>> Also when it is downloading the file it appears to the user
>>>> to be several MB long despite the file that is output being
>>>> only a few thousand KB.
>
> NB a few thousand KiB is the definition of several MiB.
>
>> I am possibly just an idiot on that one. I think the deadline
>> sleep deprivation is getting to me
It happens to all of us.
> Is the response built before you stream any of it to the client?
> Are you setting a Content-Length before you send any data? Are you
> using chunked responses?
>
>> I call a finish(req, res) method in my doPost which is where all
>> my response building is done. I am setting a content length
>> before I send the data because I wanted to be able to see how
>> long it was going to take to download since the program is fairly
>> slow.
It doesn't seem like concatenating files would take that much time,
besides the upload time. I don't know much about WAV files, though...
perhaps you have to do the math to connect them together properly. If
you have to re-encode entire files, though, you are probably doing it
wrong.
>>>> Sometimes when it doesn't work it does not produce these
>>>> errors, but it never produces these errors when it works
>>>> correctly.
>
> Does (increased) load seem to make the situation worse?
>
>> Definitely. Always happens with long audio files and only
>> sometimes with shorter test ones.
So long files end up breaking things more than short files? When I
said "load", I really meant "numbers of simultaneous requests"
(increasing).
If you can't even handle one isolated request with long files, you
have a serious problem.
>>>> I have googled it but there's no reference to the second kind
>>>> of error anywhere on the web aside from svn commits by
>>>> tomcat developers, so while I am a total newbie to mailing
>>>> lists, after exhausting stackoverflow this seemed like the
>>>> only place to turn to. I'm developing a java EE web
>>>> application in Netbeans using Tomcat 8.0.3.0 on a windows 7
>>>> operating system.
>
> (You should upgrade to a later Tomcat 8 beta. Nothing should
> affect what you are seeing here, but it's not a bad idea to get the
> latest.)
>
>> I'm just a little nervous about moving or touching anything since
>> i'm not 100% confident it wont just break everything. Still new
>> at this. I wanted the most stable version, to balance out my
>> intellectual instability.
The most stable version of 7.0.53. The 8.0.x line hasn't yet been
voted stable, though it pretty much is. Latest 8.0.x is 8.0.5 and will
soon be 8.0.6, but it has only been voted beta at this point.
>>>> The web application is very basic and allows users to upload
>>>> files via a multipart html form. This then posts to a servlet
>>>> which first uploads these files to the programs file system
>>>> and then accesses them, concatenates them together and saves
>>>> them back on the file system. It's better explained by this
>>>> diagram http://imgur.com/Oacd4gq
>
> The real question is whether you are using non-request-processor
> threads to do any of this work. Also, where are the various data
> stored while you are working on them? Not the filesystem -- that
> seems self-evident -- but where in the program are the String[]
> objects that represent all the songs that have been uploaded?
>
>> C:\Users\Leigh\Documents\NetBeansProjects\MyApp\src\java\sound
>
>> All audio files get stored in and are accessed from there.
No, somewhere in your program you have a String[] declared where you
have a reference to those filenames. You shouldn't actually have
references to path names at all, because the container can buffer the
files in memory instead of ever writing them to the disk.
Anyhow, I was wondering if you were using a String[] declared at the
servlet level to track this stuff. That always results in disaster,
because multiple requests will fight over the same variable and step
on each other's toes.
Any chance you could post your code somewhere for inspection? Dropping
it right into a post to the list is just fine.
>> This is purely a dissertation project so I have no intention of
>> making it work for multiple users or in fact be functional
>> anywhere outside of my laptop. All files are being uploaded to
>> the file system of the project itself since that was the only way
>> I knew how to make it work.
Why bother with a web application interface, then?
> When you process a "transaction" (upload -> merge -> respond), are
> you taking care to ensure that the files are placed into different
> paths for each request? How are you choosing the filename of the
> merged audio clip?
>
>> all uploads are called uploadedfile1, uploadedfile2,
>> uploadedfile3 etc depending on how many files are uploaded.
If these files all go to the same place, then multiple requests will
overwrite each other. That's going to be a problem, except you said
you don't care. Honestly, you should either care about this and make
it work properly, or just drop the "web" requirement where it's okay
to be sloppy.
>> When these files don't exist already on my file system I
>> sometimes get null pointer errors and an empty output.
>> Overwriting works better than creating new.
>
>> The filename of the finished file is just called finished.wav.
>> Always. It just gets overwritten each time I run it. Or not as it
>> seems.
I would choose a different filename every time. Especially if it looks
like the old copy of the file hasn't been closed entirely, yet.
>>>> Could it be that the files are not being uploaded fully
>>>> before they are being accessed?
>
> I don't believe this is possible when using Tomcat's file-upload
> implementation. It is certainly possible to do if you are
> performing the multipart handling yourself. What tool are you using
> to fetch the multipart data from the request?
>
>> As I said above I just get the parts from the request, for loop
>> through them and write any audio files to the locations specified
>> in my MultipartConfig.
Looking at the code would be very helpful. Feel free to leave-out any
cutting-edge research-y stuff that you don't want to divulge ;)
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTYUtlAAoJEBzwKT+lPKRYrtsQAKaQHBjvC8i4UJngL3mDtGMD
t3JxMET3Nb/AGAkeBFn2yJR+PTR4Vuj1gbEKrHN+SE3l1VlbVmjxfOQlPDAm3SPF
SE50zTnmYKKdcOMHpIuUeiMgJh/uKCym+bp3PNcuvORZq/FVJdJ3b8me0MQUO3Vh
juCqj/h/UV+eKZaFmLA+dcJAQoxmHgVoCidnwRN255LUve8/o/JQXthTjmCGuf4Z
rQ7ZarxVr8wbXZJ5wQZ/O1Z6K7IIZ07hNbDHCtj+3ZEeEcSzn5IEOMpskhSu48bC
5LTqn5gXgE7m+uFszjjpshRhoO2PPRf2sb39IYVkkExqVb5EOkdNpVVckRaBd371
H14S/CoWQcCvfq+QORl/i95vrMf+P7mMqD+wS3Bagyw3zD0spTu48oLcyG2Crmwd
zCFyezw3SIJdis8yQBsl7uGNMELF5gJ6n0oPvcUwjiY7tHbZ6f6eF5aEPAz/r1eS
uf3OHeqJswRW9qBUGPxEtvNOajs4q0o6BacfQpiQlHeLNzh6OzGsLmzUZJGoWiPV
zJR+52Z+5RFXDI38E+XSX/T9UFF7M1WjH0mfjpYz6TjIDZl7H+FH7Od5+tQ02M4C
tlr66gq2HqpQA1SlhnRpIpSsyF61PIJIteX1rcRW6qik6qizn4XJQ8GoweCwV8Jn
F2ASYC6UsDVoHpHtJfrg
=uGx4
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8.0.3.0 getting never before seen by google Illegal
State Exception. Sevlets outputting the audio output from the previous runs
of the program instead of the current run.
Posted by "Hayward, Leigh" <lh...@student.le.ac.uk>.
Hi Chris,
Thank you so much for the speedy response .
I have no idea what i'm doing formatting wise. Pretty much
just wrapping and indenting all by hand so forgive me if
it's a mess!
>________________________________________
>From: Christopher Schultz <ch...@christopherschultz.net>
>Sent: 30 April 2014 17:58
>To: Tomcat Users List
>Subject: Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State Exception. Sevlets outputting the >audio output from the previous runs of the program instead of the current run.
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>Leigh,
>On 4/30/14, 10:31 AM, Hayward, Leigh wrote:
>> My Java EE web application takes in multiple audio inputs and
>> outputs them as a single wav file via an application/octet stream.
>Like a mixer?
Like a concatenator, adds songs one after the other into one
long song. It then overlays (mixes) another file into it containing
audio cues on top of the concatenated music. But the issue is
definitely arising at the concatenation phase.
I have the audio writing to a file after each song is added in the
concatenation phase so that i can audibly debug and the
concatenator is definitely struggling to find the uploaded files.
>> It seemingly randomly works correctly (i.e. outputting the
>> correctly manipulated audio file) but sometimes, the file from a
>> previous run of the program is output and I get one of these
>> errors:
>>
>> SEVERE [http-nio-8084-exec-30]
>> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
>> The web application [/MyApp] is still processing a request that has
>> yet to finish. This is very likely to create a memory leak. You can
>> control the time allowed for requests to finish by using the
>> unloadDelay attribute of the standard Context implementation.
>>
>> and to me seemingly random numbers of these errors:
>>
>> "SEVERE [http-nio-8084-exec-87]
>> org.apache.coyote.http11.AbstractHttp11Processor.process Error
>> processing request java.lang.IllegalStateException: The resources
>> may not be accessed if they are not currently started?"
>These kinds of things are almost always due to storing of a request or
>response option in some kind of structure that survives past the end
>of a particular request.
>Can you explain how you build the response -- specifically involving
>any non-standard threading you may do?
The response is built in a method called finish which is called in
doPost like this :
finish(req, res);
In finish I make a printwriter
I set response content type to "application/octet-stream",
set the header "Content-Disposition" to
"attachment; filename=finished.wav"
I make a FileInputStream of the finished filepath
Then I set the header content length to the length of the finished file.
then I while loop the fileInputStream into the printwriter
Then I close the FIS.
I have not touched threading since it looks terrifying.
I did experiment with making the servlet singlethreaded
since I thought it could have something to do with threads
(bad practice I know) but that did nothing to help.
>> The files always upload correctly to my filesystem, but something
>> is going wrong when I try to access them in order to process them.
>What mechanism do you use for upload?
My servlet has a location set in the @multipartconfig tag
that is where all files are saved to and retrieved from.
I am getting the Parts from the request then iterating through them,
for each part that is a file upload I use
part.write("uploadedfile" + count + ".wav");
>> Also when it is downloading the file it appears to the user to be
>> several MB long despite the file that is output being only a few
>> thousand KB.
>NB a few thousand KiB is the definition of several MiB.
I am possibly just an idiot on that one.
I think the deadline sleep deprivation is getting to me
>Is the response built before you stream any of it to the client? Are
>you setting a Content-Length before you send any data? Are you using
>chunked responses?
I call a finish(req, res) method in my doPost which is where all my
response building is done.
I am setting a content length before I send the data because I wanted
to be able to see how long it was going to take to download since the
program is fairly slow.
>> Sometimes when it doesn't work it does not produce these errors,
>> but it never produces these errors when it works correctly.
>Does (increased) load seem to make the situation worse?
Definitely. Always happens with long audio files and only sometimes
with shorter test ones.
>> I have googled it but there's no reference to the second kind of
>> error anywhere on the web aside from svn commits by tomcat
>> developers, so while I am a total newbie to mailing lists, after
>> exhausting stackoverflow this seemed like the only place to turn
>> to. I'm developing a java EE web application in Netbeans using
>> Tomcat 8.0.3.0 on a windows 7 operating system.
>(You should upgrade to a later Tomcat 8 beta. Nothing should affect
>what you are seeing here, but it's not a bad idea to get the latest.)
I'm just a little nervous about moving or touching anything since i'm
not 100% confident it wont just break everything. Still new at this.
I wanted the most stable version, to balance out my intellectual
instability.
>> The web application is very basic and allows users to upload files
>> via a multipart html form. This then posts to a servlet which first
>> uploads these files to the programs file system and then accesses
>> them, concatenates them together and saves them back on the file
>> system. It's better explained by this diagram
>> http://imgur.com/Oacd4gq
>The real question is whether you are using non-request-processor
>threads to do any of this work. Also, where are the various data
>stored while you are working on them? Not the filesystem -- that seems
>self-evident -- but where in the program are the String[] objects that
>represent all the songs that have been uploaded?
C:\Users\Leigh\Documents\NetBeansProjects\MyApp\src\java\sound
All audio files get stored in and are accessed from there.
This is purely a dissertation project so I have no intention of making it
work for multiple users or in fact be functional anywhere outside of
my laptop. All files are being uploaded to the file system of the project
itself since that was the only way I knew how to make it work.
>When you process a "transaction" (upload -> merge -> respond), are you
>taking care to ensure that the files are placed into different paths
>for each request? How are you choosing the filename of the merged
>audio clip?
all uploads are called uploadedfile1, uploadedfile2, uploadedfile3 etc
depending on how many files are uploaded.
When these files don't exist already on my file system I sometimes get
null pointer errors and an empty output. Overwriting works better than
creating new.
The filename of the finished file is just called finished.wav. Always. It
just gets overwritten each time I run it. Or not as it seems.
>> Could it be that the files are not being uploaded fully before they
>> are being accessed?
>I don't believe this is possible when using Tomcat's file-upload
>implementation. It is certainly possible to do if you are performing
>the multipart handling yourself. What tool are you using to fetch the
>multipart data from the request?
As I said above I just get the parts from the request, for loop through
them and write any audio files to the locations specified in my
MultipartConfig.
>- -chris
Thanks Chris!
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1
>Comment: GPGTools - http://gpgtools.org
>Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>iQIcBAEBCAAGBQJTYSujAAoJEBzwKT+lPKRYBaoQALMHlZ44XuEeH0cmGGofOcTE
>Hc0+T101jNxEpyUTd0dvO2u03n1Fp/a+R3vglq+gp0KwmkFZOHpdhBi0XuMbdBwx
>btm1zy3c5Lico6BKh0qGWRNSqxL6RwutsLrNmi2z0N9uwHJENPl5tpPYZQXsyYEu
>IBPaqRCdAJ+aJVe6zyhyOshF1cKP3/Hd8HD3OXARUVAuOaoEiGlHoq/GFx5e/I1b
>JprF7hZumvc3QSwXH13dseTszVpWwljx1a+2/cPk3tyU+09soFAA2ee6sGF4NBr9
>c6uB1S/dYIXGcQrJcj7ICSQNM7QKNxXUbdxBIvd7Yh51ATNcTTZxshNkyYcx1IYw
>SqoojnVMd6ivXNoW2Lx8pTCSAFId0BIFh5oJdPAUf5kV1fkGQXzrUVlz/OfVhPd/
>mZXlYiuczDPz2mI9zxxzY0RTU/pB3ckXYQ7ByQE4m8+YPXo5adw+ZHmH0QeFMVlE
>khWVte3MKUIWMBPdnYKQXQp+f+Uph6yl2XHyRAZ1l/KrrTTzcuqAzsOIaW5/x85u
>a3YMDJdcHHZ+k4Uvn6w6tjZxD8TXBNXcvB0ue5rTPeOiqAeSHojwLupDFG+2XyHh
>SAc4J0FMmDwu9AZKH2mjfYuaFIpy2ncHB7FwCuYz2dBxiTfnLyOBJcSNgSj7f18a
>rAlcVsS/LEcFyS0wgGrt
>=YahV
>-----END PGP SIGNATURE-----
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
Exception. Sevlets outputting the audio output from the previous runs of
the program instead of the current run.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Leigh,
On 4/30/14, 10:31 AM, Hayward, Leigh wrote:
> My Java EE web application takes in multiple audio inputs and
> outputs them as a single wav file via an application/octet stream.
Like a mixer?
> It seemingly randomly works correctly (i.e. outputting the
> correctly manipulated audio file) but sometimes, the file from a
> previous run of the program is output and I get one of these
> errors:
>
> SEVERE [http-nio-8084-exec-30]
> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
> The web application [/MyApp] is still processing a request that has
> yet to finish. This is very likely to create a memory leak. You can
> control the time allowed for requests to finish by using the
> unloadDelay attribute of the standard Context implementation.
>
> and to me seemingly random numbers of these errors:
>
> "SEVERE [http-nio-8084-exec-87]
> org.apache.coyote.http11.AbstractHttp11Processor.process Error
> processing request java.lang.IllegalStateException: The resources
> may not be accessed if they are not currently started?"
These kinds of things are almost always due to storing of a request or
response option in some kind of structure that survives past the end
of a particular request.
Can you explain how you build the response -- specifically involving
any non-standard threading you may do?
> The files always upload correctly to my filesystem, but something
> is going wrong when I try to access them in order to process them.
What mechanism do you use for upload?
> Also when it is downloading the file it appears to the user to be
> several MB long despite the file that is output being only a few
> thousand KB.
NB a few thousand KiB is the definition of several MiB.
Is the response built before you stream any of it to the client? Are
you setting a Content-Length before you send any data? Are you using
chunked responses?
> Sometimes when it doesn't work it does not produce these errors,
> but it never produces these errors when it works correctly.
Does (increased) load seem to make the situation worse?
> I have googled it but there's no reference to the second kind of
> error anywhere on the web aside from svn commits by tomcat
> developers, so while I am a total newbie to mailing lists, after
> exhausting stackoverflow this seemed like the only place to turn
> to. I'm developing a java EE web application in Netbeans using
> Tomcat 8.0.3.0 on a windows 7 operating system.
(You should upgrade to a later Tomcat 8 beta. Nothing should affect
what you are seeing here, but it's not a bad idea to get the latest.)
> The web application is very basic and allows users to upload files
> via a multipart html form. This then posts to a servlet which first
> uploads these files to the programs file system and then accesses
> them, concatenates them together and saves them back on the file
> system. It's better explained by this diagram
> http://imgur.com/Oacd4gq
The real question is whether you are using non-request-processor
threads to do any of this work. Also, where are the various data
stored while you are working on them? Not the filesystem -- that seems
self-evident -- but where in the program are the String[] objects that
represent all the songs that have been uploaded?
When you process a "transaction" (upload -> merge -> respond), are you
taking care to ensure that the files are placed into different paths
for each request? How are you choosing the filename of the merged
audio clip?
> Could it be that the files are not being uploaded fully before they
> are being accessed?
I don't believe this is possible when using Tomcat's file-upload
implementation. It is certainly possible to do if you are performing
the multipart handling yourself. What tool are you using to fetch the
multipart data from the request?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTYSujAAoJEBzwKT+lPKRYBaoQALMHlZ44XuEeH0cmGGofOcTE
Hc0+T101jNxEpyUTd0dvO2u03n1Fp/a+R3vglq+gp0KwmkFZOHpdhBi0XuMbdBwx
btm1zy3c5Lico6BKh0qGWRNSqxL6RwutsLrNmi2z0N9uwHJENPl5tpPYZQXsyYEu
IBPaqRCdAJ+aJVe6zyhyOshF1cKP3/Hd8HD3OXARUVAuOaoEiGlHoq/GFx5e/I1b
JprF7hZumvc3QSwXH13dseTszVpWwljx1a+2/cPk3tyU+09soFAA2ee6sGF4NBr9
c6uB1S/dYIXGcQrJcj7ICSQNM7QKNxXUbdxBIvd7Yh51ATNcTTZxshNkyYcx1IYw
SqoojnVMd6ivXNoW2Lx8pTCSAFId0BIFh5oJdPAUf5kV1fkGQXzrUVlz/OfVhPd/
mZXlYiuczDPz2mI9zxxzY0RTU/pB3ckXYQ7ByQE4m8+YPXo5adw+ZHmH0QeFMVlE
khWVte3MKUIWMBPdnYKQXQp+f+Uph6yl2XHyRAZ1l/KrrTTzcuqAzsOIaW5/x85u
a3YMDJdcHHZ+k4Uvn6w6tjZxD8TXBNXcvB0ue5rTPeOiqAeSHojwLupDFG+2XyHh
SAc4J0FMmDwu9AZKH2mjfYuaFIpy2ncHB7FwCuYz2dBxiTfnLyOBJcSNgSj7f18a
rAlcVsS/LEcFyS0wgGrt
=YahV
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 8.0.3.0 getting never before seen by google Illegal
State Exception. Sevlets outputting the audio output from the previous runs
of the program instead of the current run.
Posted by "Hayward, Leigh" <lh...@student.le.ac.uk>.
I Prematurely sent that one, not even sure i had much more
to add, probably just more eternal gratitude to chris. Outlook
has completely screwed up the formatting of this email so i
don't know whats what and whats where.I'm shaking with
stress and tiredness so i'm going to go home, sleep and find
a way to force another email program onto this computer
tomorrow.
Thanks to anyone taking the time to read this,
Leigh
________________________________________
From: Hayward, Leigh <lh...@student.le.ac.uk>
Sent: 30 April 2014 15:31
To: users@tomcat.apache.org
Subject: Tomcat 8.0.3.0 getting never before seen by google Illegal State Exception. Sevlets outputting the audio output from the previous runs of the program instead of the current run.
Hello all,
My Java EE web application takes in multiple audio inputs and outputs
them as a single wav file via an application/octet stream.
It seemingly randomly works correctly (i.e. outputting the correctly
manipulated audio file) but sometimes, the file from a previous
run of the program is output and I get one of these errors:
SEVERE [http-nio-8084-exec-30] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/MyApp] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation.
and to me seemingly random numbers of these errors:
"SEVERE [http-nio-8084-exec-87] org.apache.coyote.http11.AbstractHttp11Processor.process Error processing request java.lang.IllegalStateException: The resources may not be accessed if they are not currently started?"
The files always upload correctly to my filesystem, but something is
going wrong when I try to access them in order to process them.
Also when it is downloading the file it appears to the user
to be several MB long despite the file that is output being only a few
thousand KB.
Sometimes when it doesn't work it does not produce these errors, but it
never produces these errors when it works correctly.
I have googled it but there's no reference to the second kind of error
anywhere on the web aside from svn commits by tomcat developers,
so while I am a total newbie to mailing lists, after exhausting
stackoverflow this seemed like the only place to turn to. I'm
developing a java EE web application in Netbeans using Tomcat
8.0.3.0 on a windows 7 operating system.
The web application is very basic and allows users to upload files via a
multipart html form. This then posts to a servlet which first uploads these
files to the programs file system and then accesses them, concatenates
them together and saves them back on the file system.
It's better explained by this diagram http://imgur.com/Oacd4gq
Could it be that the files are not being uploaded fully before they are
being accessed?
Sorry if this email looks like shit i'm being forced to use outlook
by my university!
Thank you for taking the time to read my post.
Best wishes to you all regardless,
Leigh
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org