You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@esme.apache.org by Ethan Jewett <es...@gmail.com> on 2009/01/27 03:14:47 UTC

RSS actions - possible bug when changing contents of feed

Hi all,

Not sure if this is the right forum, but I was very excited to get an RSS
pull of my Twitter feed set up on Richard Hirsh's Stax deployment (
http://esmecloudserverapache.dickhirsch.staxapps.net/) of the latest ESME
code.  Actions could be the killer feature of ESME.  But I'm getting ahead
of myself.

I did a little bit of refinement of the original Yahoo Pipe I was using and
went back later in the day to see how it was working, and it had stopped
working entirely.

I wondered if changing the existing values in the RSS feed that was already
set up as an action might have broken the RSS pull mechanism, so I set up
another action with the same feed, thinking that might reset things and get
me going again.  No such luck.

I then thought there might be something wrong with the structure of my new
feed.  So I set up another pipe that was exactly the same as the original
one and just now set up an action for that.  Still no luck.

At this point I'm wondering if some supporting batch program might have
crashed on the ESME stax deployment.  If so, I'm wondering if it crashed
because of the change in the feed.  This would probably be a bug, but I'd
like to confirm before I create a JIRA ticket.

Thanks,
Ethan

Re: RSS actions - possible bug when changing contents of feed

Posted by Vassil Dichev <vd...@apache.org>.
>> Credit goes to Vassil - I'm just the bearer of good news. :->
> I haven't yet found a definite connection between the Scala bug and
> the feed scenario on ESME (couldn't reproduce it in simpler cases);
> and I'm not sure it's good news :)

Now I know it's good news, because the bug has been fixed 3 days ago
in Scala 2.8 and 2.7.x. I did some tests today and couldn't reproduce
the problem for a couple of hours, whereas before it occurred within
10-15 minutes.

Replace scala.version in pom.xml from 2.7.3 to 2.7.3-SNAPSHOT for now
to test it.

Re: Memory leak?

Posted by David Pollak <fe...@gmail.com>.
There may be a memory leak.
I'm seeing memory leaks on Lift 0.10 based sites.  I'm hoping to track them
down very soon... but between the book and the ESME sprint that ends this
week... it won't likely happen this week. :-(

On Mon, Feb 2, 2009 at 12:57 AM, Hirsch, Richard <richard.hirsch@siemens.com
> wrote:

> Do we have a memory leak somewhere?
>
> Take a look at the process memory at the stax site. There are no TCP
> connections but process memory keeps on increasing.
>
> D.
>
>
>
>
>


-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Git some: http://github.com/dpp

Re: Memory leak?

Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Feb 2, 2009 at 10:18 AM, Hirsch, Richard
<ri...@siemens.com> wrote:
> Here is the screenshot (http://esmeproject.googlecode.com/files/stax%20memory.jpg)  since apache mailing lists don't allow attachments.

Some attachments are allowed on our lists, though I'm not sure about
the exact rules. http://markmail.org/message/oyifjsatt3oicspo for
example attaches at PDF document.

-Bertrand

Re: Memory leak?

Posted by Vassil Dichev <vd...@apache.org>.
Is this 100k I'm seeing? This is not yet a reason to worry- a regular
IDE consumes much more usually ;-) Maybe the application hasn't
reached a threshold level to do a full GC. Let's give it more time and
check again (the app has been running for less than a day).

> Isn't going to help us on stax where there is no file system access.

The options provided enable logging to stdout/stderr, so should work.

> Have you seen the comments on the last ESME blog (http://blog.esme.us/results-esmes-first-load-test/) and the latest stax blog (http://developer.stax.net/profiles/blogs/load-testing-on-stax) about cloud-based performance issues? I'd add your comments there.

I checked these but haven't thought much about performance yet so can't comment.

AW: Memory leak?

Posted by "Hirsch, Richard" <ri...@siemens.com>.
Isn't going to help us on stax where there is no file system access. 

Have you seen the comments on the last ESME blog (http://blog.esme.us/results-esmes-first-load-test/) and the latest stax blog (http://developer.stax.net/profiles/blogs/load-testing-on-stax) about cloud-based performance issues? I'd add your comments there.

D.   

-----Ursprüngliche Nachricht-----
Von: vdichev@gmail.com [mailto:vdichev@gmail.com] Im Auftrag von Vassil Dichev
Gesendet: Montag, 02. Februar 2009 10:49
An: esme-dev@incubator.apache.org
Betreff: Re: Memory leak?

Could you please set MAVEN_OPTS to the following and run it again:

-verbose:gc -XX:+PrintGCDetails -XX:+PrintClassHistogram

Re: Memory leak?

Posted by Vassil Dichev <vd...@apache.org>.
Could you please set MAVEN_OPTS to the following and run it again:

-verbose:gc -XX:+PrintGCDetails -XX:+PrintClassHistogram

AW: Memory leak?

Posted by "Hirsch, Richard" <ri...@siemens.com>.
Here is the screenshot (http://esmeproject.googlecode.com/files/stax%20memory.jpg)  since apache mailing lists don't allow attachments.

D.  

-----Ursprüngliche Nachricht-----
Von: Hirsch, Richard 
Gesendet: Montag, 02. Februar 2009 09:57
An: esme-dev@incubator.apache.org
Betreff: Memory leak?

Do we have a memory leak somewhere?

Take a look at the process memory at the stax site. There are no TCP
connections but process memory keeps on increasing.

D. 





Memory leak?

Posted by "Hirsch, Richard" <ri...@siemens.com>.
Do we have a memory leak somewhere?

Take a look at the process memory at the stax site. There are no TCP
connections but process memory keeps on increasing.

D. 





Re: RSS actions - possible bug when changing contents of feed

Posted by Vassil Dichev <vd...@gmail.com>.
> Credit goes to Vassil - I'm just the bearer of good news. :->
I haven't yet found a definite connection between the Scala bug and
the feed scenario on ESME (couldn't reproduce it in simpler cases);
and I'm not sure it's good news :)

RE: RSS actions - possible bug when changing contents of feed

Posted by "Hirsch, Richard" <ri...@siemens.com>.
Credit goes to Vassil - I'm just the bearer of good news. :->
 
D. 

________________________________

From: Ethan Jewett [mailto:esjewett@gmail.com]
Sent: Sun 2/1/2009 18:22
To: esme-dev@incubator.apache.org
Subject: Re: RSS actions - possible bug when changing contents of feed



Looks to me like they they think the problem is now fixed in Scala trunk.  I
assume there will be some significant delay before this fix makes it into
ESME, but I'm happy to wait.  Thanks Vassil and Richard  for tracking this
down!

Ethan

On Sun, Feb 1, 2009 at 10:37 AM, Hirsch, Richard <richard.hirsch@siemens.com
> wrote:

> Vassil found that the cause looks like it is a known Scala bug:
> http://lampsvn.epfl.ch/trac/scala/ticket/1652 <
> https://owa.siemens.at/esme/u/TG2U4V2SPO1OIZIQ;jsessionid=F917022A305A7C1D815BF1C96FDB568E
> >
>
> This is the exception:
>
>
> RSS Error details: ERROR - The ActorWatcher restarted
> org.apache.esme.actor.SchedulerActor@1fff951 because
> java.lang.IllegalStateException: Timer already cancelled.
> java.lang.IllegalStateException: Timer already cancelled. at
> java.util.Timer.sched(Timer.java:354) at
> java.util.Timer.schedule(Timer.java:170) at
> scala.actors.Actor$class.reactWithin(Actor.scala:553) at
> org.apache.esme.actor.SchedulerActor.reactWithin(SchedulerActor.scala:32) at
> org.apache.esme.actor.SchedulerActor$$anonfun$act$1.apply(SchedulerActor.scala:39)
> at
> org.apache.esme.actor.SchedulerActor$$anonfun$act$1.apply(SchedulerActor.scala:39)
> at scala.actors.Actor$class.scala$actors$Actor$$seq(Actor.scala:802) at
> scala.actors.Actor$$anon$8.andThen(Actor.scala:256) at
> scala.actors.Actor$.loop(Actor.scala:265) at
> scala.actors.Actor$$anonfun$loop$1.apply(Actor.scala:265) at
> scala.actors.Actor$$anonfun$loop$1.apply(Actor.scala:265) at
> scala.actors.Actor$$anonfun$scala$actors$Actor$$seq$1$$anonfun$apply$3.apply(Actor.scala:799)
> at
> scala.actors.Actor$$anonfun$scala$actors$Actor$$seq$1$$anonfun$apply$3.apply(Actor.scala:799)
> at scala.actors.Reaction.run(Reaction.scala:78) at
> scala.actors.Scheduler$$anon$2.run(Scheduler.scala:77)
>
> D.
>
> ________________________________
>
> From: Ethan Jewett [mailto:esjewett@gmail.com]
> Sent: Tue 1/27/2009 13:31
> To: esme-dev@incubator.apache.org
> Subject: Re: RSS actions - possible bug when changing contents of feed
>
>
>
> Twitter let me know you were working on the actions and also that Richard
> had set up a recent build on Stax.  I've been keeping an eye on it for the
> last week or so :-)
>
> I don't have an environment set up right now that would let me run trunk,
> so
> the feeds are:
>
> The Pipes feed I'm currently trying to use (the one where I changed the
> format of the title after it had been set up as an action) is
> http://pipes.yahoo.com/pipes/pipe.info?_id=fJZ29qrr3RGjLPJI6icw5g
>
> The Pipes feed that is the same as the original feed that worked once is
> http://pipes.yahoo.com/pipes/pipe.info?_id=9c4481071b4dd9c9d3619574918f1288
>
> You should be able to access both of them since they are published.
>
> Thanks for looking into this!
>
> Ethan
>
> On Tue, Jan 27, 2009 at 2:55 AM, Vassil Dichev <vd...@apache.org> wrote:
>
> > > Not sure if this is the right forum, but I was very excited to get an
> RSS
> > > pull of my Twitter feed set up on Richard Hirsh's Stax deployment (
> > > http://esmecloudserverapache.dickhirsch.staxapps.net/) of the latest
> > ESME
> > > code.  Actions could be the killer feature of ESME.  But I'm getting
> > ahead
> > > of myself.
> >
> > Yes, this is the right mailing list. It's amazing how you found out
> > about the feed actions the moment it's up.
> >
> > > I did a little bit of refinement of the original Yahoo Pipe I was using
> > and
> >
> > Very cool stuff, I'm interested to see where it goes!
> >
> > > At this point I'm wondering if some supporting batch program might have
> > > crashed on the ESME stax deployment.  If so, I'm wondering if it
> crashed
> > > because of the change in the feed.  This would probably be a bug, but
> I'd
> > > like to confirm before I create a JIRA ticket.
> >
> > It's surely a bug, but might be a known one, and maybe even fixed in
> > trunk. I did a couple of changes to the feed fetching mechanism last
> > week so that it's more flexible when the format is not as expected.
> >
> > Right now, you could either check out trunk and see if it works for
> > you or you could let me know what the feed is so that I can test it. I
> > suggest you send me the feed, since you'll probably have to do that
> > anyway if trunk doesn't work.
> >
> > Thanks for your feedback,
> > Vassil
> >
>
>
>



Re: RSS actions - possible bug when changing contents of feed

Posted by Ethan Jewett <es...@gmail.com>.
Looks to me like they they think the problem is now fixed in Scala trunk.  I
assume there will be some significant delay before this fix makes it into
ESME, but I'm happy to wait.  Thanks Vassil and Richard  for tracking this
down!

Ethan

On Sun, Feb 1, 2009 at 10:37 AM, Hirsch, Richard <richard.hirsch@siemens.com
> wrote:

> Vassil found that the cause looks like it is a known Scala bug:
> http://lampsvn.epfl.ch/trac/scala/ticket/1652 <
> https://owa.siemens.at/esme/u/TG2U4V2SPO1OIZIQ;jsessionid=F917022A305A7C1D815BF1C96FDB568E
> >
>
> This is the exception:
>
>
> RSS Error details: ERROR - The ActorWatcher restarted
> org.apache.esme.actor.SchedulerActor@1fff951 because
> java.lang.IllegalStateException: Timer already cancelled.
> java.lang.IllegalStateException: Timer already cancelled. at
> java.util.Timer.sched(Timer.java:354) at
> java.util.Timer.schedule(Timer.java:170) at
> scala.actors.Actor$class.reactWithin(Actor.scala:553) at
> org.apache.esme.actor.SchedulerActor.reactWithin(SchedulerActor.scala:32) at
> org.apache.esme.actor.SchedulerActor$$anonfun$act$1.apply(SchedulerActor.scala:39)
> at
> org.apache.esme.actor.SchedulerActor$$anonfun$act$1.apply(SchedulerActor.scala:39)
> at scala.actors.Actor$class.scala$actors$Actor$$seq(Actor.scala:802) at
> scala.actors.Actor$$anon$8.andThen(Actor.scala:256) at
> scala.actors.Actor$.loop(Actor.scala:265) at
> scala.actors.Actor$$anonfun$loop$1.apply(Actor.scala:265) at
> scala.actors.Actor$$anonfun$loop$1.apply(Actor.scala:265) at
> scala.actors.Actor$$anonfun$scala$actors$Actor$$seq$1$$anonfun$apply$3.apply(Actor.scala:799)
> at
> scala.actors.Actor$$anonfun$scala$actors$Actor$$seq$1$$anonfun$apply$3.apply(Actor.scala:799)
> at scala.actors.Reaction.run(Reaction.scala:78) at
> scala.actors.Scheduler$$anon$2.run(Scheduler.scala:77)
>
> D.
>
> ________________________________
>
> From: Ethan Jewett [mailto:esjewett@gmail.com]
> Sent: Tue 1/27/2009 13:31
> To: esme-dev@incubator.apache.org
> Subject: Re: RSS actions - possible bug when changing contents of feed
>
>
>
> Twitter let me know you were working on the actions and also that Richard
> had set up a recent build on Stax.  I've been keeping an eye on it for the
> last week or so :-)
>
> I don't have an environment set up right now that would let me run trunk,
> so
> the feeds are:
>
> The Pipes feed I'm currently trying to use (the one where I changed the
> format of the title after it had been set up as an action) is
> http://pipes.yahoo.com/pipes/pipe.info?_id=fJZ29qrr3RGjLPJI6icw5g
>
> The Pipes feed that is the same as the original feed that worked once is
> http://pipes.yahoo.com/pipes/pipe.info?_id=9c4481071b4dd9c9d3619574918f1288
>
> You should be able to access both of them since they are published.
>
> Thanks for looking into this!
>
> Ethan
>
> On Tue, Jan 27, 2009 at 2:55 AM, Vassil Dichev <vd...@apache.org> wrote:
>
> > > Not sure if this is the right forum, but I was very excited to get an
> RSS
> > > pull of my Twitter feed set up on Richard Hirsh's Stax deployment (
> > > http://esmecloudserverapache.dickhirsch.staxapps.net/) of the latest
> > ESME
> > > code.  Actions could be the killer feature of ESME.  But I'm getting
> > ahead
> > > of myself.
> >
> > Yes, this is the right mailing list. It's amazing how you found out
> > about the feed actions the moment it's up.
> >
> > > I did a little bit of refinement of the original Yahoo Pipe I was using
> > and
> >
> > Very cool stuff, I'm interested to see where it goes!
> >
> > > At this point I'm wondering if some supporting batch program might have
> > > crashed on the ESME stax deployment.  If so, I'm wondering if it
> crashed
> > > because of the change in the feed.  This would probably be a bug, but
> I'd
> > > like to confirm before I create a JIRA ticket.
> >
> > It's surely a bug, but might be a known one, and maybe even fixed in
> > trunk. I did a couple of changes to the feed fetching mechanism last
> > week so that it's more flexible when the format is not as expected.
> >
> > Right now, you could either check out trunk and see if it works for
> > you or you could let me know what the feed is so that I can test it. I
> > suggest you send me the feed, since you'll probably have to do that
> > anyway if trunk doesn't work.
> >
> > Thanks for your feedback,
> > Vassil
> >
>
>
>

RE: RSS actions - possible bug when changing contents of feed

Posted by "Hirsch, Richard" <ri...@siemens.com>.
Vassil found that the cause looks like it is a known Scala bug: http://lampsvn.epfl.ch/trac/scala/ticket/1652 <https://owa.siemens.at/esme/u/TG2U4V2SPO1OIZIQ;jsessionid=F917022A305A7C1D815BF1C96FDB568E>  
 
This is the exception:
 

RSS Error details: ERROR - The ActorWatcher restarted org.apache.esme.actor.SchedulerActor@1fff951 because java.lang.IllegalStateException: Timer already cancelled. java.lang.IllegalStateException: Timer already cancelled. at java.util.Timer.sched(Timer.java:354) at java.util.Timer.schedule(Timer.java:170) at scala.actors.Actor$class.reactWithin(Actor.scala:553) at org.apache.esme.actor.SchedulerActor.reactWithin(SchedulerActor.scala:32) at org.apache.esme.actor.SchedulerActor$$anonfun$act$1.apply(SchedulerActor.scala:39) at org.apache.esme.actor.SchedulerActor$$anonfun$act$1.apply(SchedulerActor.scala:39) at scala.actors.Actor$class.scala$actors$Actor$$seq(Actor.scala:802) at scala.actors.Actor$$anon$8.andThen(Actor.scala:256) at scala.actors.Actor$.loop(Actor.scala:265) at scala.actors.Actor$$anonfun$loop$1.apply(Actor.scala:265) at scala.actors.Actor$$anonfun$loop$1.apply(Actor.scala:265) at scala.actors.Actor$$anonfun$scala$actors$Actor$$seq$1$$anonfun$apply$3.apply(Actor.scala:799) at scala.actors.Actor$$anonfun$scala$actors$Actor$$seq$1$$anonfun$apply$3.apply(Actor.scala:799) at scala.actors.Reaction.run(Reaction.scala:78) at scala.actors.Scheduler$$anon$2.run(Scheduler.scala:77)

D. 

________________________________

From: Ethan Jewett [mailto:esjewett@gmail.com]
Sent: Tue 1/27/2009 13:31
To: esme-dev@incubator.apache.org
Subject: Re: RSS actions - possible bug when changing contents of feed



Twitter let me know you were working on the actions and also that Richard
had set up a recent build on Stax.  I've been keeping an eye on it for the
last week or so :-)

I don't have an environment set up right now that would let me run trunk, so
the feeds are:

The Pipes feed I'm currently trying to use (the one where I changed the
format of the title after it had been set up as an action) is
http://pipes.yahoo.com/pipes/pipe.info?_id=fJZ29qrr3RGjLPJI6icw5g

The Pipes feed that is the same as the original feed that worked once is
http://pipes.yahoo.com/pipes/pipe.info?_id=9c4481071b4dd9c9d3619574918f1288

You should be able to access both of them since they are published.

Thanks for looking into this!

Ethan

On Tue, Jan 27, 2009 at 2:55 AM, Vassil Dichev <vd...@apache.org> wrote:

> > Not sure if this is the right forum, but I was very excited to get an RSS
> > pull of my Twitter feed set up on Richard Hirsh's Stax deployment (
> > http://esmecloudserverapache.dickhirsch.staxapps.net/) of the latest
> ESME
> > code.  Actions could be the killer feature of ESME.  But I'm getting
> ahead
> > of myself.
>
> Yes, this is the right mailing list. It's amazing how you found out
> about the feed actions the moment it's up.
>
> > I did a little bit of refinement of the original Yahoo Pipe I was using
> and
>
> Very cool stuff, I'm interested to see where it goes!
>
> > At this point I'm wondering if some supporting batch program might have
> > crashed on the ESME stax deployment.  If so, I'm wondering if it crashed
> > because of the change in the feed.  This would probably be a bug, but I'd
> > like to confirm before I create a JIRA ticket.
>
> It's surely a bug, but might be a known one, and maybe even fixed in
> trunk. I did a couple of changes to the feed fetching mechanism last
> week so that it's more flexible when the format is not as expected.
>
> Right now, you could either check out trunk and see if it works for
> you or you could let me know what the feed is so that I can test it. I
> suggest you send me the feed, since you'll probably have to do that
> anyway if trunk doesn't work.
>
> Thanks for your feedback,
> Vassil
>



Re: RSS actions - possible bug when changing contents of feed

Posted by Ethan Jewett <es...@gmail.com>.
Twitter let me know you were working on the actions and also that Richard
had set up a recent build on Stax.  I've been keeping an eye on it for the
last week or so :-)

I don't have an environment set up right now that would let me run trunk, so
the feeds are:

The Pipes feed I'm currently trying to use (the one where I changed the
format of the title after it had been set up as an action) is
http://pipes.yahoo.com/pipes/pipe.info?_id=fJZ29qrr3RGjLPJI6icw5g

The Pipes feed that is the same as the original feed that worked once is
http://pipes.yahoo.com/pipes/pipe.info?_id=9c4481071b4dd9c9d3619574918f1288

You should be able to access both of them since they are published.

Thanks for looking into this!

Ethan

On Tue, Jan 27, 2009 at 2:55 AM, Vassil Dichev <vd...@apache.org> wrote:

> > Not sure if this is the right forum, but I was very excited to get an RSS
> > pull of my Twitter feed set up on Richard Hirsh's Stax deployment (
> > http://esmecloudserverapache.dickhirsch.staxapps.net/) of the latest
> ESME
> > code.  Actions could be the killer feature of ESME.  But I'm getting
> ahead
> > of myself.
>
> Yes, this is the right mailing list. It's amazing how you found out
> about the feed actions the moment it's up.
>
> > I did a little bit of refinement of the original Yahoo Pipe I was using
> and
>
> Very cool stuff, I'm interested to see where it goes!
>
> > At this point I'm wondering if some supporting batch program might have
> > crashed on the ESME stax deployment.  If so, I'm wondering if it crashed
> > because of the change in the feed.  This would probably be a bug, but I'd
> > like to confirm before I create a JIRA ticket.
>
> It's surely a bug, but might be a known one, and maybe even fixed in
> trunk. I did a couple of changes to the feed fetching mechanism last
> week so that it's more flexible when the format is not as expected.
>
> Right now, you could either check out trunk and see if it works for
> you or you could let me know what the feed is so that I can test it. I
> suggest you send me the feed, since you'll probably have to do that
> anyway if trunk doesn't work.
>
> Thanks for your feedback,
> Vassil
>

Re: RSS actions - possible bug when changing contents of feed

Posted by Vassil Dichev <vd...@apache.org>.
> Not sure if this is the right forum, but I was very excited to get an RSS
> pull of my Twitter feed set up on Richard Hirsh's Stax deployment (
> http://esmecloudserverapache.dickhirsch.staxapps.net/) of the latest ESME
> code.  Actions could be the killer feature of ESME.  But I'm getting ahead
> of myself.

Yes, this is the right mailing list. It's amazing how you found out
about the feed actions the moment it's up.

> I did a little bit of refinement of the original Yahoo Pipe I was using and

Very cool stuff, I'm interested to see where it goes!

> At this point I'm wondering if some supporting batch program might have
> crashed on the ESME stax deployment.  If so, I'm wondering if it crashed
> because of the change in the feed.  This would probably be a bug, but I'd
> like to confirm before I create a JIRA ticket.

It's surely a bug, but might be a known one, and maybe even fixed in
trunk. I did a couple of changes to the feed fetching mechanism last
week so that it's more flexible when the format is not as expected.

Right now, you could either check out trunk and see if it works for
you or you could let me know what the feed is so that I can test it. I
suggest you send me the feed, since you'll probably have to do that
anyway if trunk doesn't work.

Thanks for your feedback,
Vassil