Re: Joining the Apache Droids...

[Александр Зажигин <za...@mail.ru>]

Hello Thorsten,


=== Droids administration application ===

 > Droids aims to be an intelligent standalone robot framework that 
allows to create and extend existing droids (robots). In the future it 
will offer an administration application to manage and controll the 
different droids.

What is the droids module you are gonna use as the basement for such 
administration application?
Is it "droids-wicket" or "droids-crawler/droids-crawler-web" or 
something completely new?

We have started to implement such application based on Eclipse SDK.
As soon as droids is our teacher of wisdom and we would like to reuse 
maximum of droids framework concepts would be great
to know your plans according to such future application and to continue 
our current work in the same direction to be able to reuse such 
framework when it will be ready.

Such eclipse based application might be a part of droids-framework (it 
may be the eclipse mirrow for the basic droids web-based application)?
What do you think about it? (in the attachement our prototype screenshot)


=== Remote XML Task driven Architecture ===

 > Sounds good and actually the current architecture offers nearly all the
 > components that are needed. We may need to review the task interface to
 > allow xml configuration but that should not be a problem. The listener
 > approach, that a droids accept xml task via http, but is running a demon
 > on the server is the way I always saw a droid.

Is it proper understanding of the droids framework as the linux demon by 
"jsvc"?
http://commons.apache.org/daemon/jsvc.html

Do we understand the Listener approach correct by using "httpcore server 
example" and do call of droids-framework at the moment of getting the 
signal?
http://svn.apache.org/repos/asf/httpcomponents/httpcore/tags/4.0.1/httpcore/src/examples/org/apache/http/examples/ElementalHttpServer.java

One thing when I am not sure about the above "httpcore server example" 
that at the moment when we receive the XML-Task and call the specific 
droid to handle it
we have to pause of listening the port till the moment the droid will 
finish the job. Also sometimes we need to call some special functions 
(like cache cleaning and some others) and the listener will not be 
listening at this moment as well. And the questing is how to parallelize 
such three jobs: a). listening the port; b). droid does the job; c). 
doing the service fuctions. What is the solution in terms of 
Droids-Framework? How it was designed for such cases?
Again, is it "droids-wicket" or "droids-crawler/droids-crawler-web" or 
something which we aren't currently see?

 > We may need to review the task interface to allow xml configuration 
but that should not be a problem.
Could you say a few words about - where and how to review such task 
interface in terms of used java packages and must implement java classes?


=== Droids implementation Architecture ===

What is the best Droids-Framework implementation for the set of droids 
in our case (described in previous and this email)?
What is the right decision SimpleDroid (on httpcore server example), 
Wicket Application or Crawler (Crawler-Web) based?
Or something which will generalize the best of all of these?

 > Your use case sounds very interesting and our company has a similar case
 > in planing state. Droids is a framework so we are talking to extend the
 > framework with a couple of components.

We are trying to invent the best, beautiful and droids-essence solution.
How do you see your use case implementation, how do you see ours and 
where is the "golden mean" which will be the essence of droids usage?


With Best Regards,
Alexander Zazhigin.



> Hi Alexander,
>
> I replied to the mailing list since your initial mail should have gone
> to this list since it is about the Apache Droids project.
>
> On Wed, 2010-05-26 at 09:16 +0400, Александр Зажигин wrote:
> >  Hello Thorsten,
> >
> >  We are gonna use the Apache Droids architecture in our personal search
> >  project.
> >  We need some time to get info the deep understanding the concept and
> >  become the member of the committers member.
> >
> >  Right now we see few interesting issues which we are interesting to
> >  implement in the Apache Droids project.
> >  - Different Droids (with its Workers) getting corresponding XML Tasks
> >  from remote Operator (by HTTP) and sending back the XML Reports with the
> >  finished Jobs.
> >  - Such Droids are working on the different Domains (Windows / Ubuntu
> >  Servers) months without interruption and reconfiguration (it needs only
> >  XML Tasks).
> >  - Operator is preparing the Tasks and sending its by HTTP to all Domains
> >  and to all Droids and by the need sending to it Requests to receive
> >  collected Reports.
> >
> >  So, specifics will be remain from the Apache Droids project but such
> >  archtecture (framework) would possible be a part of the Apache Droids
> >  project.
>
> Your use case sounds very interesting and our company has a similar case
> in planing state. Droids is a framework so we are talking to extend the
> framework with a couple of components.
>
> >  What do you think about such implementation inside basic community
> >  version of Apache Droids?
>
> Sounds good and actually the current architecture offers nearly all the
> components that are needed. We may need to review the task interface to
> allow xml configuration but that should not be a problem. The listener
> approach, that a droids accept xml task via http, but is running a demon
> on the server is the way I always saw a droid.
>
> >  If you like this idea than we will would work hard to do the best for
> >  the project, :-)
>
> Always happy to see new people on this list and we will try our best to
> give you a helping hand.
>
> salu2
>
> >
> >  With Best Regards,
> >  Alexander Zazhigin,
> >  http://zazhigin.ru/
> >  +7 985 979 8996 mob.
> >  45464380 icq.
> >
>
> -- 
> Thorsten Scherler<thorsten.at.apache.org>
> Open Source Java<consulting, training and solutions>
>
>
>    

Re: Joining the Apache Droids...

[Thorsten Scherler <th...@apache.org>]

On 13/06/2010, at 19:26, Александр Зажигин wrote:

> Hello Thorsten,
> 
> 
> === Droids administration application ===
> 
> > Droids aims to be an intelligent standalone robot framework that allows to create and extend existing droids (robots). In the future it will offer an administration application to manage and controll the different droids.
> 
> What is the droids module you are gonna use as the basement for such administration application?
> Is it "droids-wicket" or "droids-crawler/droids-crawler-web" or something completely new?

Ryan started with the droids-wicket module as this admin application but it is still basic ATM. Bertli talked about jmx integration which would be very helpful for this task.

> 
> We have started to implement such application based on Eclipse SDK.
> As soon as droids is our teacher of wisdom and we would like to reuse maximum of droids framework concepts would be great
> to know your plans according to such future application and to continue our current work in the same direction to be able to reuse such framework when it will be ready.

If you can contribute this implementation then we can review it. I personally use eclipse and some other committer as well. Further since eclipse is java based it should be easy to reuse the code.

> 
> Such eclipse based application might be a part of droids-framework (it may be the eclipse mirrow for the basic droids web-based application)?
> What do you think about it? (in the attachement our prototype screenshot)
> 

Being an eclipse plugin or a simple web app that shows the process of the different droids to start with sounds good. the screenshots looks good but the node structure I am not 100% sure about. For example the domain node  in a crawler not always makes sense.

> 
> === Remote XML Task driven Architecture ===
> 
> > Sounds good and actually the current architecture offers nearly all the
> > components that are needed. We may need to review the task interface to
> > allow xml configuration but that should not be a problem. The listener
> > approach, that a droids accept xml task via http, but is running a demon
> > on the server is the way I always saw a droid.
> 
> Is it proper understanding of the droids framework as the linux demon by "jsvc"?
> http://commons.apache.org/daemon/jsvc.html

Yeah, more or less it is the same idea of the starting and waiting f droids and workers.

> 
> Do we understand the Listener approach correct by using "httpcore server example" and do call of droids-framework at the moment of getting the signal?
> http://svn.apache.org/repos/asf/httpcomponents/httpcore/tags/4.0.1/httpcore/src/examples/org/apache/http/examples/ElementalHttpServer.java

ATM the droids are started via java directly and the listener approach is not implemented. The link shows an example of an basic http server. Actually very similar code you will find in our junit test. Which starts a httpd server and then crawls this server. 

> 
> One thing when I am not sure about the above "httpcore server example" that at the moment when we receive the XML-Task and call the specific droid to handle it
> we have to pause of listening the port till the moment the droid will finish the job. Also sometimes we need to call some special functions (like cache cleaning and some others) and the listener will not be listening at this moment as well. And the questing is how to parallelize such three jobs: a). listening the port; b). droid does the job; c). doing the service fuctions. What is the solution in terms of Droids-Framework? How it was designed for such cases?
> Again, is it "droids-wicket" or "droids-crawler/droids-crawler-web" or something which we aren't currently see?

ATM the above described architecture is ATM not implemented. However I reckon that would go into droids-wicket where you upload an xml-task definition. Then a parser would read this task and generate droids tasks and invoke the responsible droids. the monitoring application would e.g. use jmx to display the process.


> 
> > We may need to review the task interface to allow xml configuration but that should not be a problem.
> Could you say a few words about - where and how to review such task interface in terms of used java packages and must implement java classes?
> 

Actually depends on the xml definition you want to use. Please see the api what you can do atm with it but your usecase maybe needs to expose more methods.

> 
> === Droids implementation Architecture ===
> 
> What is the best Droids-Framework implementation for the set of droids in our case (described in previous and this email)?
> What is the right decision SimpleDroid (on httpcore server example), Wicket Application or Crawler (Crawler-Web) based?
> Or something which will generalize the best of all of these?

Like described above you can start with the wicket module and add the monitoring facility to it. That is just the component to  monitor but you still need to create your own droids that will do the tasks.

> 
> > Your use case sounds very interesting and our company has a similar case
> > in planing state. Droids is a framework so we are talking to extend the
> > framework with a couple of components.
> 
> We are trying to invent the best, beautiful and droids-essence solution.
> How do you see your use case implementation, how do you see ours and where is the "golden mean" which will be the essence of droids usage?

You need to be aware that a lot you are trying to do is on our list of that things we want to implement, however most of them are not implemented yet, but there you will enter. ;)

salu2
 


Thorsten Scherler <thorsten.at.apache.org>
Open Source Java <consulting, training and solutions>