You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Jan Høydahl (Created JIRA)" <ji...@apache.org> on 2011/10/16 15:00:12 UTC
[jira] [Created] (SOLR-2841) Scriptable UpdateRequestChain
Scriptable UpdateRequestChain
-----------------------------
Key: SOLR-2841
URL: https://issues.apache.org/jira/browse/SOLR-2841
Project: Solr
Issue Type: New Feature
Components: update
Reporter: Jan Høydahl
UpdateProcessorChains must currently be defined with XML in solrconfig.xml. We should explore a scriptable chain implementation with a DSL that allows for full flexibility. The first step would be to make UpdateChain implementations pluggable in solrconfig.xml, for backward compat support.
Benefits and possibilities with a Scriptable UpdateChain:
* A compact DSL for defining Processors and Chains (Workflows would be a better, less limited term here)
* Keeping update processor config separate from solrconfig.xml gives better separations of roles
* Use this as an opportunity to natively support scripting language Processors (ideas from SOLR-1725)
This issue is spun off from SOLR-2823.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2841) Scriptable UpdateRequestChain
Posted by "Jan Høydahl (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174742#comment-13174742 ]
Jan Høydahl commented on SOLR-2841:
-----------------------------------
Interesting idea, but only update side has been considered this far.
> Scriptable UpdateRequestChain
> -----------------------------
>
> Key: SOLR-2841
> URL: https://issues.apache.org/jira/browse/SOLR-2841
> Project: Solr
> Issue Type: New Feature
> Components: update
> Reporter: Jan Høydahl
>
> UpdateProcessorChains must currently be defined with XML in solrconfig.xml. We should explore a scriptable chain implementation with a DSL that allows for full flexibility. The first step would be to make UpdateChain implementations pluggable in solrconfig.xml, for backward compat support.
> Benefits and possibilities with a Scriptable UpdateChain:
> * A compact DSL for defining Processors and Chains (Workflows would be a better, less limited term here)
> * Keeping update processor config separate from solrconfig.xml gives better separations of roles
> * Use this as an opportunity to natively support scripting language Processors (ideas from SOLR-1725)
> This issue is spun off from SOLR-2823.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2841) Scriptable UpdateRequestChain
Posted by "Lance Norskog (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174566#comment-13174566 ]
Lance Norskog commented on SOLR-2841:
-------------------------------------
+1
Another use case for scripting at the top level is "multi-query" queries: where the app creates the second based on the first. Would your proposal handle this problem?
Many use cases for grouping/collapsing can be implemented with 2 queries. Perhaps the guts of collapsing could be simplified if the more outré use cases could be pushed out into multiple queries.
> Scriptable UpdateRequestChain
> -----------------------------
>
> Key: SOLR-2841
> URL: https://issues.apache.org/jira/browse/SOLR-2841
> Project: Solr
> Issue Type: New Feature
> Components: update
> Reporter: Jan Høydahl
>
> UpdateProcessorChains must currently be defined with XML in solrconfig.xml. We should explore a scriptable chain implementation with a DSL that allows for full flexibility. The first step would be to make UpdateChain implementations pluggable in solrconfig.xml, for backward compat support.
> Benefits and possibilities with a Scriptable UpdateChain:
> * A compact DSL for defining Processors and Chains (Workflows would be a better, less limited term here)
> * Keeping update processor config separate from solrconfig.xml gives better separations of roles
> * Use this as an opportunity to natively support scripting language Processors (ideas from SOLR-1725)
> This issue is spun off from SOLR-2823.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2841) Scriptable UpdateRequestChain
Posted by "Chris Male (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174747#comment-13174747 ]
Chris Male commented on SOLR-2841:
----------------------------------
bq. Another use case for scripting at the top level is "multi-query" queries: where the app creates the second based on the first. Would your proposal handle this problem?
So you mean having scriptable query time logic? That seems a pretty scary thing to get into.
> Scriptable UpdateRequestChain
> -----------------------------
>
> Key: SOLR-2841
> URL: https://issues.apache.org/jira/browse/SOLR-2841
> Project: Solr
> Issue Type: New Feature
> Components: update
> Reporter: Jan Høydahl
>
> UpdateProcessorChains must currently be defined with XML in solrconfig.xml. We should explore a scriptable chain implementation with a DSL that allows for full flexibility. The first step would be to make UpdateChain implementations pluggable in solrconfig.xml, for backward compat support.
> Benefits and possibilities with a Scriptable UpdateChain:
> * A compact DSL for defining Processors and Chains (Workflows would be a better, less limited term here)
> * Keeping update processor config separate from solrconfig.xml gives better separations of roles
> * Use this as an opportunity to natively support scripting language Processors (ideas from SOLR-1725)
> This issue is spun off from SOLR-2823.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2841) Scriptable UpdateRequestChain
Posted by "Jan Høydahl (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128966#comment-13128966 ]
Jan Høydahl commented on SOLR-2841:
-----------------------------------
The DSL could be based on Groovy, JRuby, Jython or JS. Here's my quasi sketch of a Groovy example from 2823:
...This approach also solves another wish of mine, namely being able to define chains outside of solrconfig.xml. Logically, configuring schema and document processing is done by a "content" guy, but configuring solrconfig.xml is done by the "hardware/operations" guys. Imagine a solr/conf/pipeline.groovy defined in solrconfig.xml:
{code:xml}
<updateProcessorChain class="solr.ScriptedUpdateProcessorChainFactory" file="updateprocessing.groovy" />
{code}
updateprocessing.groovy:
{code}
chain simple {
process(langid)
process(copyfield)
chain(logAndRun)
}
chain moreComplex {
process(langid)
if(doc.getFieldValue("employees") > 10)
process(copyfield)
else
chain(myOtherProcesses)
doc.deleteField("title")
chain(logAndRun)
}
chain logAndRun {
process(log)
process(run)
}
processor langid {
class = "solr.LanguageIdentifierUpdateProcessorFactory"
config("langid.fl", "title,body")
config("langid.langField", "language")
config("map", true)
}
processor copyfield {
script = "copyfield.groovy"
config("from", "title")
config("to", "title_en")
}
{code}
I don't know what it takes to code such a thing, but if we had it, I'd never go back to defining pipelines in XML :)
> Scriptable UpdateRequestChain
> -----------------------------
>
> Key: SOLR-2841
> URL: https://issues.apache.org/jira/browse/SOLR-2841
> Project: Solr
> Issue Type: New Feature
> Components: update
> Reporter: Jan Høydahl
>
> UpdateProcessorChains must currently be defined with XML in solrconfig.xml. We should explore a scriptable chain implementation with a DSL that allows for full flexibility. The first step would be to make UpdateChain implementations pluggable in solrconfig.xml, for backward compat support.
> Benefits and possibilities with a Scriptable UpdateChain:
> * A compact DSL for defining Processors and Chains (Workflows would be a better, less limited term here)
> * Keeping update processor config separate from solrconfig.xml gives better separations of roles
> * Use this as an opportunity to natively support scripting language Processors (ideas from SOLR-1725)
> This issue is spun off from SOLR-2823.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org