[jira] [Commented] (NUTCH-961) Expose Tika's boilerpipe support

["Nguyen Manh Tien (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788911#comment-13788911 ] 

Nguyen Manh Tien commented on NUTCH-961:
----------------------------------------

I used patch NUTCH-961-2.1-v2.patch for nutch-2.2.1
i found that the text parsed by nutch-tika (with boilerpipe support) is different from text parsed by demo site http://boilerpipe-web.appspot.com 
I did upgrade to boilerpipe 1.2.0 to be match with demo site.

The url i tested is http://www.medhelp.org/posts/Eye-Care/EYE/show/1199003

The text from nutch-tika (i use ArticleExtractor)

EYE - Eye Care - MedHelp Experts My MedHelp Login or Signup Eye Care Community EYE Post a Question « Back to Community About This Community: This patient support community is for discussions relating to eye care, cataracts , glaucoma , retinal detachment , eye infections, misaligned eyes , intra-ocular implants, refractive surgery ( LASIK and CK), glasses, contact lenses, amblyopia , eye injuries, dry eyes , ocular allergy, eye pain and discomfort, pediatric eye disorders, eyelid and tearduct surgery, poor eyesight, and eye surgery. View community archives Font Size: A A ABackground: Search this Community: Go 3 Comments EYE My son is 4 and half years old and have + no .Our doctor told me six months ago that + no. decreases as time passed and he not to wear glasses after two -three years if he wears glasses regularly.But yesterday he told me that his + No. increases and he have to wear glasses always.If you wish u can go for laser surgery after 14 years i.e. when my son will have age of 17 years.please help me what to do ? Watch this discussion Tweet Related Discussions How to decide if glasses are needed for children? (8 replies):How can a Doctor tell if a child has amblyopia? Is t... [more] Astigmatism (1 replies):My 5 year old son has severe astigmatism. He wears glass... [more] Can someone help me in regards to my sons eyes? (6 replies):I had noticed my son had, had an eye issue when he was a... [more] Blurred vision with glasses (2 replies):Hi, I recently got new glasses and but the vision in my ... [more] Eyesight getting worse (2 replies):Hello! So here's the story. My eyesight had never been ... [more]

AND from demo

3 Comments
EYE
My son is 4 and half years old and have + no .Our doctor told me six months ago that + no. decreases as time passed and he not to wear glasses after two -three years if he wears glasses regularly.But yesterday he told me that his + No. increases and he have to wear glasses always.If you wish u can go for laser surgery after 14 years i.e. when my son will have age of 17 years.please help me what to do ?

the result from demo is much better for this url.
So the parse-tike/boilerpipe not only extract main content from page but also include title and other node content.
Is it expected?

> Expose Tika's boilerpipe support
> --------------------------------
>
>                 Key: NUTCH-961
>                 URL: https://issues.apache.org/jira/browse/NUTCH-961
>             Project: Nutch
>          Issue Type: New Feature
>          Components: parser
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>             Fix For: 2.3, 1.8
>
>         Attachments: BoilerpipeExtractorRepository.java, NUTCH-961-1.3-3.patch, NUTCH-961-1.3-tikaparser1.patch, NUTCH-961-1.3-tikaparser.patch, NUTCH-961-1.4-dombuilder-1.patch, NUTCH-961-1.5-1.patch, NUTCH-961-1.8-1.patch, NUTCH-961-2.1-v1.patch, NUTCH-961-2.1-v2.patch, NUTCH-961v2.patch
>
>
> Tika 0.8 comes with the Boilerpipe content handler which can be used to extract boilerplate content from HTML pages. We should see how we can expose Boilerplate in the Nutch cofiguration.



--
This message was sent by Atlassian JIRA
(v6.1#6144)