You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "David Stuart (JIRA)" <ji...@apache.org> on 2010/10/22 12:28:15 UTC
[jira] Created: (NUTCH-924) Static field in solr mapping
Static field in solr mapping
----------------------------
Key: NUTCH-924
URL: https://issues.apache.org/jira/browse/NUTCH-924
Project: Nutch
Issue Type: Improvement
Components: indexer
Affects Versions: 1.3
Reporter: David Stuart
Fix For: 1.3
Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-924) Static field in solr mapping
Posted by "David Stuart (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Stuart updated NUTCH-924:
-------------------------------
Attachment: nutch_1.3_static_field.patch
updated with formattng, think I need eclipse code styles
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Assignee: Markus Jelsma
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch, nutch_1.3_static_field.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-924) Static field in solr mapping
Posted by "David Stuart (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Stuart updated NUTCH-924:
-------------------------------
Attachment: nutch_1.3_static_field.patch
Extends SolrWriter and SolrMappingReader to take static field data defined in solrindex-mapping.xml
e.g.
<staticField dest="entity">node</staticField>
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-924) Static field in solr mapping
Posted by "David Stuart (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Stuart updated NUTCH-924:
-------------------------------
Attachment: nutch_1.3_static_field2.patch
I have reworked the patch slightly to accomplish the following
1. adds a single static value to a specified field
2. adds another value to a field already mapped by Nutch (requires multi valued field)
3. The ability to define multiple staticField's with the same dest attribute will create multiple entires (requires multi valued field)
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Assignee: Markus Jelsma
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch, nutch_1.3_static_field.patch, nutch_1.3_static_field2.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (NUTCH-924) Static field in solr mapping
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma reassigned NUTCH-924:
-----------------------------------
Assignee: Markus Jelsma
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Assignee: Markus Jelsma
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-924) Static field in solr mapping
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923845#action_12923845 ]
Andrzej Bialecki commented on NUTCH-924:
-----------------------------------------
The functionality is useful, +1. But the patch has formatting errors. Please fix them before committing.
The same functionality should be added to trunk, too.
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Assignee: Markus Jelsma
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (NUTCH-924) Static field in solr mapping
Posted by Markus Jelsma <ma...@openindex.io>.
Yes. As it already adds a second value for an existing NutchDocument field, it
makes sense that it also would allow for multiple staticField definitions with
the same dest parameter.
I'm unsure how to do this with the used HashMap object as its put method
overwrites any previous value.
> So im clear the patch is all good except to extend it allow for
> specification of multiple staticField's with the same dest attribute this
> would (given a multiple value field) populate said field?
>
> Regards
>
> On 22 Oct 2010, at 14:56, Markus Jelsma (JIRA) wrote:
> > [
> > https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.ji
> > ra.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1292
> > 3861#action_12923861 ]
> >
> > Markus Jelsma commented on NUTCH-924:
> > -------------------------------------
> >
> > Great! The patch almost works as i expected. It:
> > 1. adds a single static value to a specified field
> > 2. adds another value to a field already mapped by Nutch (requires multi
> > valued field) 3. adds the last defined staticField value (doesn't
> > require a multi valued field).
> >
> > I think 1 and 2 are desired, this way we can append values to fields. But
> > i think 3 isn't a desirable feature. If we can use this to append a
> > value for a multi valued field, then shouldn't it also work if we define
> > multiple staticField elements with the same dest attribute?
> >
> >> Static field in solr mapping
> >> ----------------------------
> >>
> >> Key: NUTCH-924
> >> URL: https://issues.apache.org/jira/browse/NUTCH-924
> >>
> >> Project: Nutch
> >>
> >> Issue Type: Improvement
> >> Components: indexer
> >>
> >> Affects Versions: 1.3
> >>
> >> Reporter: David Stuart
> >> Assignee: Markus Jelsma
> >>
> >> Fix For: 1.3
> >>
> >> Attachments: nutch_1.3_static_field.patch,
> >> nutch_1.3_static_field.patch
> >>
> >> Original Estimate: 0h
> >>
> >> Remaining Estimate: 0h
> >>
> >> Provide the facility to pass static data defined in
> >> solrindex-mapping.xml to solr during the mapping process.
Re: [jira] Commented: (NUTCH-924) Static field in solr mapping
Posted by David Stuart <da...@progressivealliance.co.uk>.
So im clear the patch is all good except to extend it allow for specification of multiple staticField's with the same dest attribute this would (given a multiple value field) populate said field?
Regards
On 22 Oct 2010, at 14:56, Markus Jelsma (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923861#action_12923861 ]
>
> Markus Jelsma commented on NUTCH-924:
> -------------------------------------
>
> Great! The patch almost works as i expected. It:
> 1. adds a single static value to a specified field
> 2. adds another value to a field already mapped by Nutch (requires multi valued field)
> 3. adds the last defined staticField value (doesn't require a multi valued field).
>
> I think 1 and 2 are desired, this way we can append values to fields. But i think 3 isn't a desirable feature. If we can use this to append a value for a multi valued field, then shouldn't it also work if we define multiple staticField elements with the same dest attribute?
>
>> Static field in solr mapping
>> ----------------------------
>>
>> Key: NUTCH-924
>> URL: https://issues.apache.org/jira/browse/NUTCH-924
>> Project: Nutch
>> Issue Type: Improvement
>> Components: indexer
>> Affects Versions: 1.3
>> Reporter: David Stuart
>> Assignee: Markus Jelsma
>> Fix For: 1.3
>>
>> Attachments: nutch_1.3_static_field.patch, nutch_1.3_static_field.patch
>>
>> Original Estimate: 0h
>> Remaining Estimate: 0h
>>
>> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (NUTCH-924) Static field in solr mapping
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923861#action_12923861 ]
Markus Jelsma commented on NUTCH-924:
-------------------------------------
Great! The patch almost works as i expected. It:
1. adds a single static value to a specified field
2. adds another value to a field already mapped by Nutch (requires multi valued field)
3. adds the last defined staticField value (doesn't require a multi valued field).
I think 1 and 2 are desired, this way we can append values to fields. But i think 3 isn't a desirable feature. If we can use this to append a value for a multi valued field, then shouldn't it also work if we define multiple staticField elements with the same dest attribute?
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Assignee: Markus Jelsma
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch, nutch_1.3_static_field.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (NUTCH-924) Static field in solr mapping
Posted by David Stuart <da...@progressivealliance.co.uk>.
Just updating the formatting in the patch now and will resubmit
On 22 Oct 2010, at 14:25, Markus Jelsma (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923851#action_12923851 ]
>
> Markus Jelsma commented on NUTCH-924:
> -------------------------------------
>
> Yes, i'll look into it next week orso. The pro for this patch is that in some cases it can replace the subcollection plugin to set a static value.
>
>> Static field in solr mapping
>> ----------------------------
>>
>> Key: NUTCH-924
>> URL: https://issues.apache.org/jira/browse/NUTCH-924
>> Project: Nutch
>> Issue Type: Improvement
>> Components: indexer
>> Affects Versions: 1.3
>> Reporter: David Stuart
>> Assignee: Markus Jelsma
>> Fix For: 1.3
>>
>> Attachments: nutch_1.3_static_field.patch
>>
>> Original Estimate: 0h
>> Remaining Estimate: 0h
>>
>> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (NUTCH-924) Static field in solr mapping
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923851#action_12923851 ]
Markus Jelsma commented on NUTCH-924:
-------------------------------------
Yes, i'll look into it next week orso. The pro for this patch is that in some cases it can replace the subcollection plugin to set a static value.
> Static field in solr mapping
> ----------------------------
>
> Key: NUTCH-924
> URL: https://issues.apache.org/jira/browse/NUTCH-924
> Project: Nutch
> Issue Type: Improvement
> Components: indexer
> Affects Versions: 1.3
> Reporter: David Stuart
> Assignee: Markus Jelsma
> Fix For: 1.3
>
> Attachments: nutch_1.3_static_field.patch
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Provide the facility to pass static data defined in solrindex-mapping.xml to solr during the mapping process.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.