You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Jettro Coenradie <je...@gridshore.nl> on 2010/08/16 13:37:26 UTC

Connector architecture question and suggestion

Hi,
I am having a look at the connectors. At the moment to my opinion the
classes for (all) connectors are to big. This is partly due to the way the
interfaces are structured and partly due to the implementation of html in
java. For example the RssConnector now has almost 6000 lines of code and the
JdbcConnector has almost 2000 lines of code. To my opinion this can be
improved by making separate components for presenting and configuring the
connectors in the crawler-ui web project and for the part needed by the
runner. Abstracting the html from the actual classes can help a lot as well.
Maybe some utility methods to make creating these html pages easier is nice
as well.

I am willing to investigate this path further, but I'd like to have ideas of
other developers. It would be nice to know if others feel like this can be
improved as well.

It might be interesting to look at a technique like wicket for the ui part.
Than you can package the html code together with the java code in one jar.
No difficult repackaging is required and you can still create nice
interfaces. I also read that others want to have a look at something like
velocity, of course that can be a valid option as well.

So what is your opinions about it?

regards

Jettro Coenradie
http://www.gridshore.nl

Re: Connector architecture question and suggestion

Posted by Jettro Coenradie <je...@gridshore.nl>.
Oke, I will come up with a proposal for breaking up the components in a way
that still enables us to easily keep the current adapters in their own
structure.

As for the UI testen, it is always a beast. We do have some experience with
tools like Webdriver and others. Will have a look at that as well.

On Mon, Aug 16, 2010 at 2:41 PM, <ka...@nokia.com> wrote:

> I think that providing tools/help for implementing the UI pieces of
> connectors is a perfectly reasonable thing to do.  However, I strongly
> believe that the UI components should remain described as part of the
> connector interfaces.  Breaking their implementations out within individual
> connectors is also reasonable, but unless there is a compelling reason to
> refactor all the individual connectors in that way, I would hold off that
> project until there are better unit tests for the connector UI components.
> If you are willing to contribute such tests, I think going that way would be
> worth consideration.
>
> I'd like to see a more detailed proposal before I comment further.
>
> Thanks,
> Karl
>
> -----Original Message-----
> From: jettro.coenradie@gmail.com [mailto:jettro.coenradie@gmail.com] On
> Behalf Of ext Jettro Coenradie
> Sent: Monday, August 16, 2010 7:37 AM
> To: connectors-dev@incubator.apache.org
> Subject: Connector architecture question and suggestion
>
> Hi,
> I am having a look at the connectors. At the moment to my opinion the
> classes for (all) connectors are to big. This is partly due to the way the
> interfaces are structured and partly due to the implementation of html in
> java. For example the RssConnector now has almost 6000 lines of code and
> the
> JdbcConnector has almost 2000 lines of code. To my opinion this can be
> improved by making separate components for presenting and configuring the
> connectors in the crawler-ui web project and for the part needed by the
> runner. Abstracting the html from the actual classes can help a lot as
> well.
> Maybe some utility methods to make creating these html pages easier is nice
> as well.
>
> I am willing to investigate this path further, but I'd like to have ideas
> of
> other developers. It would be nice to know if others feel like this can be
> improved as well.
>
> It might be interesting to look at a technique like wicket for the ui part.
> Than you can package the html code together with the java code in one jar.
> No difficult repackaging is required and you can still create nice
> interfaces. I also read that others want to have a look at something like
> velocity, of course that can be a valid option as well.
>
> So what is your opinions about it?
>
> regards
>
> Jettro Coenradie
> http://www.gridshore.nl
>



-- 
Jettro Coenradie
http://www.gridshore.nl

RE: Connector architecture question and suggestion

Posted by ka...@nokia.com.
I think that providing tools/help for implementing the UI pieces of connectors is a perfectly reasonable thing to do.  However, I strongly believe that the UI components should remain described as part of the connector interfaces.  Breaking their implementations out within individual connectors is also reasonable, but unless there is a compelling reason to refactor all the individual connectors in that way, I would hold off that project until there are better unit tests for the connector UI components. If you are willing to contribute such tests, I think going that way would be worth consideration.

I'd like to see a more detailed proposal before I comment further.

Thanks,
Karl

-----Original Message-----
From: jettro.coenradie@gmail.com [mailto:jettro.coenradie@gmail.com] On Behalf Of ext Jettro Coenradie
Sent: Monday, August 16, 2010 7:37 AM
To: connectors-dev@incubator.apache.org
Subject: Connector architecture question and suggestion

Hi,
I am having a look at the connectors. At the moment to my opinion the
classes for (all) connectors are to big. This is partly due to the way the
interfaces are structured and partly due to the implementation of html in
java. For example the RssConnector now has almost 6000 lines of code and the
JdbcConnector has almost 2000 lines of code. To my opinion this can be
improved by making separate components for presenting and configuring the
connectors in the crawler-ui web project and for the part needed by the
runner. Abstracting the html from the actual classes can help a lot as well.
Maybe some utility methods to make creating these html pages easier is nice
as well.

I am willing to investigate this path further, but I'd like to have ideas of
other developers. It would be nice to know if others feel like this can be
improved as well.

It might be interesting to look at a technique like wicket for the ui part.
Than you can package the html code together with the java code in one jar.
No difficult repackaging is required and you can still create nice
interfaces. I also read that others want to have a look at something like
velocity, of course that can be a valid option as well.

So what is your opinions about it?

regards

Jettro Coenradie
http://www.gridshore.nl