You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Magnus Ebbesson (JIRA)" <ji...@apache.org> on 2012/05/18 15:04:06 UTC
[jira] [Created] (CONNECTORS-475) A Hydra Output Connector
Magnus Ebbesson created CONNECTORS-475:
------------------------------------------
Summary: A Hydra Output Connector
Key: CONNECTORS-475
URL: https://issues.apache.org/jira/browse/CONNECTORS-475
Project: ManifoldCF
Issue Type: New Feature
Reporter: Magnus Ebbesson
Priority: Minor
Hydra Processing Framework was recently released into the wild.
Hydra offers to solve the the missing piece into creating great consolidated search solutions.
What is Hydra?
When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
More information and code to the framework on
https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CONNECTORS-475) A Hydra Output Connector
Posted by "Erlend Garåsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erlend Garåsen reassigned CONNECTORS-475:
-----------------------------------------
Assignee: Erlend Garåsen
> A Hydra Output Connector
> ------------------------
>
> Key: CONNECTORS-475
> URL: https://issues.apache.org/jira/browse/CONNECTORS-475
> Project: ManifoldCF
> Issue Type: New Feature
> Components: Framework agents process
> Reporter: Magnus Ebbesson
> Assignee: Erlend Garåsen
> Priority: Minor
> Labels: output, outputconnector
>
> Hydra Processing Framework was recently released into the wild.
> Hydra offers to solve the the missing piece into creating great consolidated search solutions.
> What is Hydra?
> When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
> Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
> More information and code to the framework on
> https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-475) A Hydra Output Connector
Posted by "Magnus Ebbesson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Magnus Ebbesson updated CONNECTORS-475:
---------------------------------------
Component/s: Framework agents process
> A Hydra Output Connector
> ------------------------
>
> Key: CONNECTORS-475
> URL: https://issues.apache.org/jira/browse/CONNECTORS-475
> Project: ManifoldCF
> Issue Type: New Feature
> Components: Framework agents process
> Reporter: Magnus Ebbesson
> Priority: Minor
> Labels: output, outputconnector
>
> Hydra Processing Framework was recently released into the wild.
> Hydra offers to solve the the missing piece into creating great consolidated search solutions.
> What is Hydra?
> When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
> Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
> More information and code to the framework on
> https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-475) A Hydra Output Connector
Posted by "Erlend Garåsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292723#comment-13292723 ]
Erlend Garåsen commented on CONNECTORS-475:
-------------------------------------------
Developing a Hydra output connector will probably be valuable for the community. I have assigned the issue to me in order to learn more about the Hydra connector which was presented at Lucene Revolution 2012 in Boston.
It seems that there are some problems with dependencies which are required in order to build Hydra core. According to the development history, it seems that the following activity removed these dependencies:
> Removed Findwise depencensies and made sure every module compiles [ebbesson]
If I try to compile Hydra by using Maven, I get these error messages. Could you please explain how to get around these problems, either by a comment here or send me an e-mail at ridder (at) apache.org? You may write in Swedish in case you prefer the latter.
{code}
[ERROR] Failed to execute goal on project hydra-core: Could not resolve dependencies for project com.findwise.hydra:hydra-core:jar:0.2.0-SNAPSHOT: The following artifacts could not be resolved: com.findwise.hydra:hydra-api:jar:0.2.0-SNAPSHOT, com.findwise.hydra:hydra-database:jar:0.2.0-SNAPSHOT: ...
{code}
> A Hydra Output Connector
> ------------------------
>
> Key: CONNECTORS-475
> URL: https://issues.apache.org/jira/browse/CONNECTORS-475
> Project: ManifoldCF
> Issue Type: New Feature
> Components: Framework agents process
> Reporter: Magnus Ebbesson
> Assignee: Erlend Garåsen
> Priority: Minor
> Labels: output, outputconnector
>
> Hydra Processing Framework was recently released into the wild.
> Hydra offers to solve the the missing piece into creating great consolidated search solutions.
> What is Hydra?
> When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
> Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
> More information and code to the framework on
> https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-475) A Hydra Output Connector
Posted by "Magnus Ebbesson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Magnus Ebbesson updated CONNECTORS-475:
---------------------------------------
Labels: output outputconnector (was: )
> A Hydra Output Connector
> ------------------------
>
> Key: CONNECTORS-475
> URL: https://issues.apache.org/jira/browse/CONNECTORS-475
> Project: ManifoldCF
> Issue Type: New Feature
> Reporter: Magnus Ebbesson
> Priority: Minor
> Labels: output, outputconnector
>
> Hydra Processing Framework was recently released into the wild.
> Hydra offers to solve the the missing piece into creating great consolidated search solutions.
> What is Hydra?
> When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
> Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
> More information and code to the framework on
> https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-475) A Hydra Output Connector
Posted by "Erlend Garåsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400435#comment-13400435 ]
Erlend Garåsen commented on CONNECTORS-475:
-------------------------------------------
I managed to build Hydra with some help from Joel Westberg from Findwise. I needed to do the following in the correct order:
Start MongoDB
cd api
mvn install
cd ../database
mvn install
cd ../core
mvn install
> A Hydra Output Connector
> ------------------------
>
> Key: CONNECTORS-475
> URL: https://issues.apache.org/jira/browse/CONNECTORS-475
> Project: ManifoldCF
> Issue Type: New Feature
> Components: Framework agents process
> Reporter: Magnus Ebbesson
> Assignee: Erlend Garåsen
> Priority: Minor
> Labels: output, outputconnector
>
> Hydra Processing Framework was recently released into the wild.
> Hydra offers to solve the the missing piece into creating great consolidated search solutions.
> What is Hydra?
> When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
> Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
> More information and code to the framework on
> https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-475) A Hydra Output Connector
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400442#comment-13400442 ]
Karl Wright commented on CONNECTORS-475:
----------------------------------------
Please let me know when you are ready to work on the UI parts of the connector. I've updated ManifoldCF in Action to cover usage of Velocity templates, which you'd probably find helpful.
> A Hydra Output Connector
> ------------------------
>
> Key: CONNECTORS-475
> URL: https://issues.apache.org/jira/browse/CONNECTORS-475
> Project: ManifoldCF
> Issue Type: New Feature
> Components: Framework agents process
> Reporter: Magnus Ebbesson
> Assignee: Erlend Garåsen
> Priority: Minor
> Labels: output, outputconnector
>
> Hydra Processing Framework was recently released into the wild.
> Hydra offers to solve the the missing piece into creating great consolidated search solutions.
> What is Hydra?
> When working with free text search using for example Apache Solr the quality of the data in the index is a key factor of the quality of the results delivered. Hydra is designed to give the search solution the tools necessary to modify the data that is to be indexed in an efficient and flexible way. This is done by providing a scalable and efficient pipeline which the documents will have to pass through before being indexed into the search engine.
> Architecturally Hydra sits in between the search engine and the source integration. A common use-case would be to use Apache Manifold CF to crawl a folder on a filesystem and send the documents to hydra which in turn will process and dispatch processed documents to Solr for indexing.
> More information and code to the framework on
> https://github.com/Findwise/Hydra
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira