You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Karl Wright (JIRA)" <ji...@apache.org> on 2010/06/02 14:47:42 UTC
[jira] Created: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Classloader-based plug-in architecture would permit LCF to be prebuilt
----------------------------------------------------------------------
Key: CONNECTORS-40
URL: https://issues.apache.org/jira/browse/CONNECTORS-40
Project: Lucene Connector Framework
Issue Type: Improvement
Components: Framework core
Reporter: Karl Wright
The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884285#action_12884285 ]
Karl Wright commented on CONNECTORS-40:
---------------------------------------
Classloader has bee added, and the configuration file format is now XML. The wiki connector description pages have been updated. Next:
- Change the build process and connector delivery model to take advantage of the classloader
- Change the build process wiki document to reflect all changes
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
> Assignee: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CONNECTORS-40) Classloader-based
plug-in architecture would permit LCF to be prebuilt
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879107#action_12879107 ]
Mark Miller edited comment on CONNECTORS-40 at 6/15/10 7:32 PM:
----------------------------------------------------------------
FYI: http://search.lucidimagination.com/search/document/1b60f001fc0a0a98/branch_for_ticket_connectors_40
This issue is being developed on a branch at http://svn.apache.org/repos/asf/incubator/lcf/branches/CONNECTORS-40
was (Author: markrmiller@gmail.com):
FYI: http://search.lucidimagination.com/search/document/1b60f001fc0a0a98/branch_for_ticket_connectors_40
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883595#action_12883595 ]
Karl Wright commented on CONNECTORS-40:
---------------------------------------
The UI changes have been made, largely hand-tested, and merged into trunk. Next steps for this ticket include:
- Updating the wiki page on how to build a connector
- Writing the classloader implementation that will actually allow for plugin loading
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879178#action_12879178 ]
Mark Miller commented on CONNECTORS-40:
---------------------------------------
>From http://search.lucidimagination.com/search/document/760aeaa785116e3b/beginning_of_connectors_40_work :
Hi all (and especially Eric),
I began work on CONNECTORS-40 in the agreed-upon branch. So far, I've checked in the modifications needed to pull output connector UI out of JSP, and also did the conversion of the gts output connector from JSP. This looks reasonably good to me, other than the somewhat-more-obtuse syntax required to represent HTML from within the java connector classes. But it would be good to hear any comments before I go further in the conversion process.
Thanks,
Karl
Mark: you can find a link to the diffs ref'd here: http://mail-archives.apache.org/mod_mbox/incubator-connectors-commits/201006.mbox/%3C20100615191345.6A20723888E3@eris.apache.org%3E
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879174#action_12879174 ]
Mark Miller commented on CONNECTORS-40:
---------------------------------------
Hey Karl -
To lower the bar for contributers (say those with ideas but not the time to jump into code), could you summarize what you are up to with this issue?
It also is difficult to casually follow along with this issue especially as it is broken up over the mailing list, a branch, and JIRA.
Ideally everything one would want to follow along would be in the JIRA issue itself. In that spirit, I'll post some more useful info.
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879182#action_12879182 ]
Mark Miller commented on CONNECTORS-40:
---------------------------------------
bq. could you summarize what you are up to with this issue?
To clarify, I mean could you summarize the approach you are taking with the current work you have done on the branch.
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Erik Hatcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879193#action_12879193 ]
Erik Hatcher commented on CONNECTORS-40:
----------------------------------------
At first glance, this looks like a great first step towards ridding LCF of JSPs.
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-40.
-----------------------------------
Resolution: Fixed
All code committed. Related tickets (such as removing the need for connector-specific -D switches) still in progress.
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
> Assignee: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879107#action_12879107 ]
Mark Miller commented on CONNECTORS-40:
---------------------------------------
FYI: http://search.lucidimagination.com/search/document/1b60f001fc0a0a98/branch_for_ticket_connectors_40
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879215#action_12879215 ]
Karl Wright commented on CONNECTORS-40:
---------------------------------------
The implementation strategy is as follows:
(1) Add methods to the connector interfaces to support the UI. These correspond directly to the chunks of UI contributed by each connector that used to be performed by jsps, which used to be located by a naming technique. (Every connector had a family of jsps, e.g. "output/<connector_name>/headerconfig.jsp", "output/<connector_name>/editconfig.jsp", etc.) To do this in a way that will make it possible to easily replace the technology for the framework side of the UI later, I also introduced some interfaces so that there are no direct references to any JSP or servlet classes.
(2) Change the framework UI to call the connector methods rather than the old jsp components.
(3) Change all individual connectors to discard their JSPs and instead implement the connector methods.
Once this preliminary work is done, it should be possible to write a class loader to allow a user (or an installer) to specify a set of paths in which to search for jars. This would make it possible for people to deliver connectors into the system without having to rebuild the war file, which currently is necessary. That, in turn, makes it feasible to prebuild all LCF components and deliver it much like Solr is delivered.
The CONNECTORS-40 branch currently contains just the following:
- UI method additions to the output connection interface only;
- Changes to the framework UI code to call the new methods;
- Changes to the GTS output connector to implement the new methods (and remove the old JSPs).
The reason this has been checked in at this point is largely as a sanity check. It's a lot easier to change direction when one connector has been done than it would be to change 15 of them.
Hope this helps.
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CONNECTORS-40) Classloader-based plug-in
architecture would permit LCF to be prebuilt
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright reassigned CONNECTORS-40:
-------------------------------------
Assignee: Karl Wright
> Classloader-based plug-in architecture would permit LCF to be prebuilt
> ----------------------------------------------------------------------
>
> Key: CONNECTORS-40
> URL: https://issues.apache.org/jira/browse/CONNECTORS-40
> Project: Lucene Connector Framework
> Issue Type: Improvement
> Components: Framework core
> Reporter: Karl Wright
> Assignee: Karl Wright
>
> The LCF architecture at this point requires interaction with the build script in order to add connectors. This is because the connector JSPs and jars need to be added to the appropriate war files. However, there is another architectural option that would eliminate this need, which is to use a custom classloader to pull components from jars that are placed in a specific directory or directories.
> In order for this to work, however, the UI components of every connector must become part of a jar. That implies that they will need to cease being JSPs, and become instead methods of each connector class. (There is no proscription against using something like Velocity for assembling the necessary output for a connector, however.) Limiting the backwards-compatibility impact of this change will be difficult, especially after a first release is made, so it seems clear that any change along these lines should be attempted before version 1.0 is released.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.