[jira] [Commented] (NUTCH-1570) Add filtering capability to Datastore Queries

["Lewis John McGibbney (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993987#comment-13993987 ] 

Lewis John McGibbney commented on NUTCH-1570:
---------------------------------------------

+1 [~jnioche]

> Add filtering capability to Datastore Queries
> ---------------------------------------------
>
>                 Key: NUTCH-1570
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1570
>             Project: Nutch
>          Issue Type: Improvement
>          Components: REST_api, storage
>    Affects Versions: 2.2
>            Reporter: Lewis John McGibbney
>             Fix For: 2.4
>
>
> For some time this issue has been discussed on various lists.
> When doing the upgrade of the Gora dependencies in NUTCH-1569, I  stumbled across a comment within o.a.n.api.DbReader#Iterator
> {code}
>   public Iterator<Map<String,Object>> iterator(String[] fields, String startKey, String endKey,
>       String batchId) throws Exception {
>     Query<String,WebPage> q = store.newQuery();
>     String[] qFields = fields;
>     if (fields != null) {
>       HashSet<String> flds = new HashSet<String>(Arrays.asList(fields));
>       // remove "url"
>       flds.remove("url");
>       if (flds.size() > 0) {
>         qFields = flds.toArray(new String[flds.size()]);
>       } else {
>         qFields = null;
>       }
>     }
>     q.setFields(qFields);
>     if (startKey != null) {
>       q.setStartKey(startKey);
>       if (endKey != null) {
>         q.setEndKey(endKey);
>       }
>     }
>     Result<String,WebPage> res = store.execute(q);
>     *XXX we should add the filtering capability to Query*
>     return new DbIterator(res, fields, batchId);
>   }
> {code} 
> I will link this issue to something over on Gora once we get around to the implementation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)