You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tika.apache.org by Koorosh Vakhshoori <Ko...@synopsys.com> on 2011/06/20 23:29:19 UTC
Excluding part of document when parsing.
I am parsing a PDF document for the purpose of indexing it in Solr, specially Solr Cell. My problem is to exclude certain areas of document from being indexed; for example Copyright section. The same argument applies to HTML pages where I don't want to index footer or header or other irrelevant sections. I looked at the MatchingContentHandler API which uses XPATH. However, the XPathParser supports a very limited set of XPATH features. Ideally, I want to use an XPATH such as:
//xhtml:body/xhtml:div\[not(contains(p,'EXCLUDE TEXT'))\]
I am thinking maybe I should customize Tika to handle these cases, any suggestion where I should start? What are my options here?
Any help is greatly appreciated.
Re: Excluding part of document when parsing.
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Jun 20, 2011 at 11:29 PM, Koorosh Vakhshoori
<Ko...@synopsys.com> wrote:
> I am parsing a PDF document for the purpose of indexing it in Solr, specially Solr Cell. My problem is to
> exclude certain areas of document from being indexed; for example Copyright section. The same
> argument applies to HTML pages where I don't want to index footer or header or other irrelevant
> sections.
Sounds like a useful feature, though currently the only thing we have
along those lines is the Boilerpipe support for HTML.
As a general rule it'll probably be easiest to build such exclusion
rules into the formats-specific parsers (for example it's easy to
exclude headers and footers within the office format parsers).
BR,
Jukka Zitting