[jira] [Comment Edited] (GORA-119) implement a filter enabled scan in gora

["Nguyen Manh Tien (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GORA-119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831172#comment-13831172 ] 

Nguyen Manh Tien edited comment on GORA-119 at 11/25/13 9:03 AM:
-----------------------------------------------------------------

Sure, i will merge Enis change.
I used this filter feature for Nutch,  it reduce the time to scan the whole hbase table in map task from 80 min to 40 min in most of crawling job. The hbase table size is 20M urls and my batch have about 100k url


was (Author: tiennm):
Sure, i will merge Enis change.
I used this filter feature for Nutch,  it reduce the time to scan the whole hbase table in map task from 90 min to 30 min in most of crawling job. The hbase table size is 20M urls and my batch have about 100k url

> implement a filter enabled scan in gora
> ---------------------------------------
>
>                 Key: GORA-119
>                 URL: https://issues.apache.org/jira/browse/GORA-119
>             Project: Apache Gora
>          Issue Type: Improvement
>    Affects Versions: 0.2
>         Environment: gora hbase gora-core gora-hbase
>            Reporter: raf shin
>              Labels: filter, gora-core, gora-hbase, scan
>             Fix For: 0.4
>
>         Attachments: GORA-119-v1.txt, gora-119-v1.1.patch, gora-119_v2.patch
>
>
> it'll be very of help to implement a filtered scan to reduce the time of scan in gora-core and gora-hbase components.



--
This message was sent by Atlassian JIRA
(v6.1#6144)