You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Yonik Seeley (Created) (JIRA)" <ji...@apache.org> on 2012/03/15 18:29:37 UTC
[jira] [Created] (SOLR-3251) dynamically add field to schema
dynamically add field to schema
-------------------------------
Key: SOLR-3251
URL: https://issues.apache.org/jira/browse/SOLR-3251
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269802#comment-13269802 ]
Grant Ingersoll commented on SOLR-3251:
---------------------------------------
I'm a big fan of restlet and it could be a nice segue to supporting more things as pure REST. Downside is another moving part. I think Restlet even has some (old) Solr integration.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-3251) dynamically add field to
schema
Posted by "Sami Siren (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230399#comment-13230399 ]
Sami Siren edited comment on SOLR-3251 at 3/15/12 6:32 PM:
-----------------------------------------------------------
bq. Any ideas for an external API?
I like the latter option more.
was (Author: siren):
I like the latter option more.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Yonik Seeley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230373#comment-13230373 ]
Yonik Seeley commented on SOLR-3251:
------------------------------------
Any ideas for an external API?
We could use a single entry point for all things schema related...
http://localhost:8983/solr/schema
{"addField":{"myfield":{"type":"int" ...}}
Or more specific to fields...
http://localhost:8983/solr/fields
OR
PUT/POST to http://localhost:8983/solr/schema/fields (nesting all schema related stuff under "schema" would help pollute the namespace less)
{"myfield":{"type":"int" ...}}
I'm leaning toward the last option. Thoughts?
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230627#comment-13230627 ]
Hoss Man commented on SOLR-3251:
--------------------------------
bq. Any ideas for an external API?
I think the best way to support this externally is using the existing mechanism for plugins...
* a RequestHandler people can register (if they want to support external clients programaticly modifying the schema) that accepts ContentStreams containing whatever payload structure makes sense given the functionality.
* an UpdateProcessor people can register (if they want to support stuff like SOLR-3250 where clients adding documents can submit any field name and a type is added based on the type of hte value) which could be configured with mappings of java types to fieldTypes and rules about other field attributes -- ie "if a client submits a new field=value with a java.lang.Integer value, create a new "tint" field with that name and set stored=true.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Sami Siren (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230399#comment-13230399 ]
Sami Siren commented on SOLR-3251:
----------------------------------
I like the latter option more.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269704#comment-13269704 ]
Yonik Seeley commented on SOLR-3251:
------------------------------------
Regarding PUT, it doesn't seem to be allowed by our current implementation (I think it's a request parser implementation detail).
Should we change that to allow us to be more REST-like?
Or should we go further and integrate something like restlet?
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Ryan McKinley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230381#comment-13230381 ]
Ryan McKinley commented on SOLR-3251:
-------------------------------------
Does this imply that the schema would be writeable?
The PUT/POST option is nicer
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230659#comment-13230659 ]
Hoss Man commented on SOLR-3251:
--------------------------------
a low level implementation detail i would worry about is "snapshoting" the schema for the duration of a single request .. i suspect there are more then a few places in solr that would generate weird exceptions if multiple calls to "req.getSchema().getFields()" returned different things in the middle of processing a single request.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230379#comment-13230379 ]
Grant Ingersoll commented on SOLR-3251:
---------------------------------------
I like the last option, b/c you can easily see this evolving to support other things like field types, etc. If we go down this route, would be nice to make it RESTful.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-3251) dynamically add field to schema
Posted by "Yonik Seeley (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-3251:
-------------------------------
Attachment: SOLR-3251.patch
Here's a quick start... no tests or external API yet.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Ryan McKinley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230390#comment-13230390 ]
Ryan McKinley commented on SOLR-3251:
-------------------------------------
What are the thoughts on error handling? are you only able to add fields that don't exist? If they exist in the schema but not in the index? What about if the index Analyzer is identical, but the query Analyzer has changed?
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Ryan McKinley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230406#comment-13230406 ]
Ryan McKinley commented on SOLR-3251:
-------------------------------------
bq. separate file alongside schema.xml
This makes sense.
As is, the ad-hoc naming conventions in schema make writing out the full schema pretty daunting.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Yonik Seeley (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230393#comment-13230393 ]
Yonik Seeley commented on SOLR-3251:
------------------------------------
bq. Does this imply that the schema would be writeable?
The in-memory schema object yes.
The question is how to persist changes. I was thinking it might be easiest to keep a separate file alongside schema.xml for dynamically added fields for now. The term "dynamicFields" has already been taken though and we probably shouldn't overload it. Maybe extra_fields.json? Or maybe even schema.json/schema.yaml that acts as an extension of schema.xml (and could acquire additional features over time such as the ability to define types too?)
But a separate file that just lists fields will be much quicker (and easier) to update. Reloading a full schema.xml (along with type instantiation) would currently be somewhat prohibitive.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-3251) dynamically add field to schema
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269699#comment-13269699 ]
Yonik Seeley commented on SOLR-3251:
------------------------------------
bq. a low level implementation detail i would worry about is "snapshoting" the schema for the duration of a single request .. i suspect there are more then a few places in solr that would generate weird exceptions if multiple calls to "req.getSchema().getFields()" returned different things in the middle of processing a single request.
It's good to think about, but I'm not sure it will be a problem in practice. Adding a new field shouldn't be an issue for most code.
Removing a field is a different matter... but if a query explicitly references a field (for example) and then it disappears, having that cause an exception is fine if it would also cause an exception if the field were missing.
Instead of snapshotting, I think we should think about where fields changing could be a problem and then harden the code against that. If it does get too difficult, then we could revisit schema snapshots.
> dynamically add field to schema
> -------------------------------
>
> Key: SOLR-3251
> URL: https://issues.apache.org/jira/browse/SOLR-3251
> Project: Solr
> Issue Type: New Feature
> Reporter: Yonik Seeley
> Attachments: SOLR-3251.patch
>
>
> One related piece of functionality needed for SOLR-3250 is the ability to dynamically add a field to the schema.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org