Report

[Richard Frovarp <rf...@apache.org>]

Our report is due to day. I haven't seen the notification message, but 
it to should have been out today. Below is what I have prepped to send 
as a report. Please send any changes to me. Due to the large number of 
projects that have to report and the very late notice, the Wiki is going 
to be a bit active, so I'm doing this via email.

Droids is an Incubator project arrived from Apache Labs. Droids entered 
incubation on October, 2008.

It's an intelligent standalone robot framework that allows one to create 
and extend existing web robots.

Richard Frovarp has been added as a mentor to the project.

IP clearance has been taken care of and was done prior to the release 
that was performed. That issue should not have been in the last report.

Issues before graduation :

The biggest obstacle for graduation is obtaining a diverse group of 
active committers. 95% of the commits over the past 6 months were by a 
single committer. The project needs to engage the committers that exist 
and identify new people to bring into the project. The goal for the next 
quarter is to elicit more activity.

Re: Report

[Thorsten Scherler <th...@apache.org>]

On 02/01/2012 05:37 PM, Richard Frovarp wrote:
> Our report is due to day. I haven't seen the notification message, but 
> it to should have been out today. Below is what I have prepped to send 
> as a report. Please send any changes to me. Due to the large number of 
> projects that have to report and the very late notice, the Wiki is 
> going to be a bit active, so I'm doing this via email.
>
> Droids is an Incubator project arrived from Apache Labs. Droids 
> entered incubation on October, 2008.
>
> It's an intelligent standalone robot framework that allows one to 
> create and extend existing web robots.
>
> Richard Frovarp has been added as a mentor to the project.
>
> IP clearance has been taken care of and was done prior to the release 
> that was performed. That issue should not have been in the last report.
>
> Issues before graduation :
>
> The biggest obstacle for graduation is obtaining a diverse group of 
> active committers. 95% of the commits over the past 6 months were by a 
> single committer. The project needs to engage the committers that 
> exist and identify new people to bring into the project. The goal for 
> the next quarter is to elicit more activity.

+1

Thanks for taking care of it.

I am thinking for a while now to incorporate an ActiveMQ queue as 
replacement of the current one into droids. This will change a bit the 
worker specification and IMO as well the overall flow through the app.

In activeMq you have listener and notifier. So e.g. a link/task 
extractor would post to a specific queue on a server/cluster. Here you 
can post as well to different queue this is helpful if you have a 
multiple steps involved prior of extracting the final information.

e.g. you need to crawl various categories where the links to the final 
information are listed. This would need 3 different handlers:
1) extract categories listing links
2) extract final information listing links
3) handle the final information
Now you could actually create 3 different worker as listener for 3 
different queues leaving each of them pretty lightweight. Further with 
activeMq the queue is cluster ready by design and the notifier can be 
implemented in any given language that can post to a REST service, be it 
ruby, perl, ... so e.g. the crawling/ part could be implemented via web 
form, rss feeds, ...

Further I have worked quite a bit with cocoon3 lately and have to admit 
that the pipelines in Java are doing 80% of the use cases with droids.

It is time for me again to get active here again.

-- 
Thorsten Scherler<thorsten.at.apache.org>
codeBusters S.L. - web based systems
<consulting, training and solutions>
http://www.codebusters.es/