You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avalon.apache.org by Berin Loritsch <bl...@apache.org> on 2002/10/26 22:12:49 UTC
Re: org.apache.excalibur.event.command package
Alex Treyger wrote:
> Hi Berin and Peter,
>
> My name is Alex; I am working on a project that is
> similar to the event, but my commands can't be
> processed quicker than in 10-20 sec. Therefore, I need
> to have more then one thread working from the same
> queue. Also, I can't use dequeueAll(), since this
> would serialize command processing.
You do have more than one thread working from the same
queue. The ThreadManager has a pool of threads to work
with. You can set it up to have a certain ratio of threads
to processors. That way if you have eight threads per
processor set, and you have two processors, the commands
will be executed within sixteen threads.
The default is one thread per processor--which on a one
processor machine is only one thread.
> To achieve my goals, I moved ImmediateRunner from
> AbstractThreadManager to outside, and introduced a few
> interfaces. My new classes, that are versions of
> yours, I put in sub-package medium.
What are you wanting to accomplish with the interfaces?
> It make more sense to me to pass Runner (with pipeline
> inside) instead of Pipeline to ThreadManager, since
> the latter one does not care about pipelines; it needs
> to execute something, that' all. For compatibility, I
> did not remove
>
> register( EventPipeline pipeline )
>
> method.
No. The reason is quite simple. The EventPipeline does
not have to be a CommandManager. It can be anything. It
stems from the SEDA package, and is focussed on event
pipelining.
Please post any questions, etc. to either the user list
or the developer list so I don't have to answer the same
questions over and over.
> All these new classes, modified classes, as well as
> the rest of classes from the package, are in a tar.gz
>
> Please, let me know what you think.
>
>
> Thanks,
> Alex Treyger
>
> PS.
>
> 1. In case you like the change, feel free to improve
> or rename classes.
>
> 2. I do not know policy, so I added my name in all
> classes I modified.
>
> 3. I added
>
> throw new RuntimeException(e)
>
> in a catch after
>
> ( (Command)element ).execute();
>
> in CommandManager.
Why? where do you expect that RuntimeException to be caught?
Are you sure that it doesn't kill the thread that is monitoring
the queues?
> 4. Should CommandManager be renamed to a
> CommandProcessor?
I was following an XXXmanager idiom. It could be, but I really
don't see the advantage.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: org.apache.excalibur.event.command package
Posted by Berin Loritsch <bl...@apache.org>.
Alex Treyger wrote:
> In my environment, it takes time and IO for a command
> to be processed.
> This is why I would like to use multiple threads to
> run different commands from the same source
> simultaneously. For this reason, I need commands to be
> pulled from a Source one in a time.
>
> To achieve this, I would like to register multiple
> instances of a Pipeline (all using the same Source
> object) with one AbstractThreadManager.
First thing:
You don't work with an **AbstractThreadManager**, you
work with a concrete implementation.
Second thing:
What is your Thread per CPU ratio?
The TPCThreadManager will collect all the Commands and
execute them one at a time using the threads from a pool.
If you have 8 threads per CPU, that means you will have
eight commands being processed (per CPU) at once. That
is by design.
> I have two problems in a AbstractThreadManager as it
> is now:
>
> 1. PipelineRunner.run()
>
> pulls all commands in one call:
>
> handler.handleEvents( sources[ i ].dequeueAll() );
Take a look at how CommandManager works with them.
> I would prefer to be able to use a Runner that pulls
> only one command in a time. This would allow another
> threads to pull and process another command from the
> same source at the same time. Of course, it makes
> sense only in a case when it takes time to process
> command, and only if such commands wait for IO.
You do not have to start with AbstractThreadManager.
Someone else made the Abstract class from my concrete
one.
> 2. register() uses PipelineRunner (which pulls all
> commands from Source in one call), and there is no
> easy way to use another Runner instead; a different
> Runner is needed for the reason #1.
Sure. Use another base class. I do get your point
though. We need to be able to use both approaches.
> Runner interface, that I introduced, has two
> additional methods:
>
> Object getType();
>
> so that all pipelines that use the same Source, can be
> deregistered in one call by type.
Honestly, I don't like that. We don't operate by type,
but by pipeline.
> boolean isAvailable();
>
> Using this call in AbstractThreadManager may prevent
> running the same runner on multiple threads (i.e.
> using more then one thread for the same Runner in case
> when it runs for a long-long time):
<snip/>
You don't need to run the same runner on multiple threads.
You need to run the Commands in multiple threads. That is
*already done*.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: org.apache.excalibur.event.command package
Posted by Alex Treyger <al...@yahoo.com>.
In my environment, it takes time and IO for a command
to be processed.
This is why I would like to use multiple threads to
run different commands from the same source
simultaneously. For this reason, I need commands to be
pulled from a Source one in a time.
To achieve this, I would like to register multiple
instances of a Pipeline (all using the same Source
object) with one AbstractThreadManager.
I have two problems in a AbstractThreadManager as it
is now:
1. PipelineRunner.run()
pulls all commands in one call:
handler.handleEvents( sources[ i ].dequeueAll() );
I would prefer to be able to use a Runner that pulls
only one command in a time. This would allow another
threads to pull and process another command from the
same source at the same time. Of course, it makes
sense only in a case when it takes time to process
command, and only if such commands wait for IO.
2. register() uses PipelineRunner (which pulls all
commands from Source in one call), and there is no
easy way to use another Runner instead; a different
Runner is needed for the reason #1.
Runner interface, that I introduced, has two
additional methods:
Object getType();
so that all pipelines that use the same Source, can be
deregistered in one call by type.
boolean isAvailable();
Using this call in AbstractThreadManager may prevent
running the same runner on multiple threads (i.e.
using more then one thread for the same Runner in case
when it runs for a long-long time):
while( i.hasNext() )
{
Runner nextRunner =
(Runner)i.next();
ThreadControl control = null;
// This does not prevent, but
often avoids
// running the same runner on
multiple threads
// INSERTED if
if( nextRunner.isAvailable() )
{
while( control == null )
{
try
{
control =
m_threadPool.execute( nextRunner );
}
catch(
IllegalStateException e )
{
// that's the way
ResourceLimitingThreadPool reports
// that it has no
threads available, will still try
// to go on,
hopefully at one point there will be
// a thread to
execute our runner
if(
getLogger().isWarnEnabled() )
{
getLogger().warn( "Unable to execute runner (If out of
threads, "
+ "increase block-timeout or number of threads "
+ "per processor", e );
}
}
}
m_controls.add( control );
}
}
Thanks,
Alex
--- Berin Loritsch <bl...@apache.org> wrote:
> Alex Treyger wrote:
> > Hi Berin and Peter,
> >
> > My name is Alex; I am working on a project that is
> > similar to the event, but my commands can't be
> > processed quicker than in 10-20 sec. Therefore, I
> need
> > to have more then one thread working from the same
> > queue. Also, I can't use dequeueAll(), since this
> > would serialize command processing.
>
> You do have more than one thread working from the
> same
> queue. The ThreadManager has a pool of threads to
> work
> with. You can set it up to have a certain ratio of
> threads
> to processors. That way if you have eight threads
> per
> processor set, and you have two processors, the
> commands
> will be executed within sixteen threads.
>
> The default is one thread per processor--which on a
> one
> processor machine is only one thread.
>
> > To achieve my goals, I moved ImmediateRunner from
> > AbstractThreadManager to outside, and introduced a
> few
> > interfaces. My new classes, that are versions of
> > yours, I put in sub-package medium.
>
> What are you wanting to accomplish with the
> interfaces?
>
> > It make more sense to me to pass Runner (with
> pipeline
> > inside) instead of Pipeline to ThreadManager,
> since
> > the latter one does not care about pipelines; it
> needs
> > to execute something, that' all. For
> compatibility, I
> > did not remove
> >
> > register( EventPipeline pipeline )
> >
> > method.
>
> No. The reason is quite simple. The EventPipeline
> does
> not have to be a CommandManager. It can be
> anything. It
> stems from the SEDA package, and is focussed on
> event
> pipelining.
>
> Please post any questions, etc. to either the user
> list
> or the developer list so I don't have to answer the
> same
> questions over and over.
>
>
> > All these new classes, modified classes, as well
> as
> > the rest of classes from the package, are in a
> tar.gz
> >
> > Please, let me know what you think.
> >
> >
> > Thanks,
> > Alex Treyger
> >
> > PS.
> >
> > 1. In case you like the change, feel free to
> improve
> > or rename classes.
> >
> > 2. I do not know policy, so I added my name in all
> > classes I modified.
> >
> > 3. I added
> >
> > throw new RuntimeException(e)
> >
> > in a catch after
> >
> > ( (Command)element ).execute();
> >
> > in CommandManager.
>
> Why? where do you expect that RuntimeException to
> be caught?
> Are you sure that it doesn't kill the thread that is
> monitoring
> the queues?
>
>
> > 4. Should CommandManager be renamed to a
> > CommandProcessor?
>
> I was following an XXXmanager idiom. It could be,
> but I really
> don't see the advantage.
>
>
> --
>
> "They that give up essential liberty to obtain a
> little temporary safety
> deserve neither liberty nor safety."
> - Benjamin Franklin
>
__________________________________________________
Do you Yahoo!?
Y! Web Hosting - Let the expert host your web site
http://webhosting.yahoo.com/
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>