You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by dm...@apache.org on 2014/03/27 10:46:54 UTC
svn commit: r1582236 [11/14] - in /tika/site: publish/ publish/0.10/
publish/0.5/ publish/0.6/ publish/0.7/ publish/0.8/ publish/0.9/
publish/1.0/ publish/1.1/ publish/1.2/ publish/1.3/ publish/1.4/
publish/1.5/ src/site/
Modified: tika/site/publish/1.3/formats.html
URL: http://svn.apache.org/viewvc/tika/site/publish/1.3/formats.html?rev=1582236&r1=1582235&r2=1582236&view=diff
==============================================================================
--- tika/site/publish/1.3/formats.html (original)
+++ tika/site/publish/1.3/formats.html Thu Mar 27 09:46:52 2014
@@ -45,7 +45,7 @@
}
}
if (provider == "lucid") {
- form.action = "http://search.lucidimagination.com/p:tika";
+ form.action = "http://find.searchhub.org/p:tika";
} else if (provider == "sl") {
form.action = "http://search-lucene.com/tika";
}
@@ -84,7 +84,68 @@
width="387" height="100"/></a>
</div>
<div id="content">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Supported Document Formats<a name="Supported_Document_Formats"></a></h2><p>This page lists all the do
cument formats supported by Apache Tika 0.6. Follow the links to the various parser class javadocs for more detailed information about each document format and how it is parsed by Tika.</p><ul><li><a href="#Supported_Document_Formats">Supported Document Formats</a><ul><li><a href="#HyperText_Markup_Language">HyperText Markup Language</a></li><li><a href="#XML_and_derived_formats">XML and derived formats</a></li><li><a href="#Microsoft_Office_document_formats">Microsoft Office document formats</a></li><li><a href="#OpenDocument_Format">OpenDocument Format</a></li><li><a href="#Portable_Document_Format">Portable Document Format</a></li><li><a href="#Electronic_Publication_Format">Electronic Publication Format</a></li><li><a href="#Rich_Text_Format">Rich Text Format</a></li><li><a href="#Compression_and_packaging_formats">Compression and packaging formats</a></li><li><a href="#Text_formats">Text formats</a></li><li><a href="#Audio_formats">Audio formats</a></li><li><a href="#Image_form
ats">Image formats</a></li><li><a href="#Video_formats">Video formats</a></li><li><a href="#Java_class_files_and_archives">Java class files and archives</a></li><li><a href="#The_mbox_format">The mbox format</a></li></ul></li></ul><div class="section"><h3><a name="HyperText_Markup_Language">HyperText Markup Language</a></h3><p>The HyperText Markup Language (HTML) is the lingua franca of the web. Tika uses the <a class="externalLink" href="http://home.ccil.org/~cowan/XML/tagsoup/">TagSoup</a> library to support virtually any kind of HTML found on the web. The output from the <a href="#apiorgapachetikaparserhtmlHtmlParser.html">HtmlParser</a> class is guaranteed to be well-formed and valid XHTML, and various heuristics are used to prevent things like inline scripts from cluttering the extracted text content.</p></div><div class="section"><h3><a name="XML_and_derived_formats">XML and derived formats</a></h3><p>The Extensible Markup Language (XML) format is a generic format that can be
used for all kinds of content. Tika has custom parsers for some widely used XML vocabularies like XHTML, OOXML and ODF, but the default <a href="#apiorgapachetikaparserxmlDcXMLParser.html">DcXMLParser</a> class simply extracts the text content of the document and ignores any XML structure. The only exception to this rule are Dublin Core metadata elements that are used for the document metadata.</p></div><div class="section"><h3><a name="Microsoft_Office_document_formats">Microsoft Office document formats</a></h3><p>Microsoft Office and some related applications produce documents in the generic OLE 2 Compound Document and Office Open XML (OOXML) formats. The older OLE 2 format was introduced in Microsoft Office version 97 and was the default format until Office version 2007 and the new XML-based OOXML format. The <a href="#apiorgapachetikaparsermicrosoftOfficeParser.html">OfficeParser</a> and <a href="#apiorgapachetikaparsermicrosoftooxmlOOXMLParser.html">OOXMLParser</a> classes use
<a class="externalLink" href="http://poi.apache.org/">Apache POI</a> libraries to support text and metadata extraction from both OLE2 and OOXML documents.</p></div><div class="section"><h3><a name="OpenDocument_Format">OpenDocument Format</a></h3><p>The OpenDocument format (ODF) is used most notably as the default format of the OpenOffice.org office suite. The <a href="#apiorgapachetikaparserodfOpenDocumentParser.html">OpenDocumentParser</a> class supports this format and the earlier OpenOffice 1.0 format on which ODF is based.</p></div><div class="section"><h3><a name="Portable_Document_Format">Portable Document Format</a></h3><p>The <a href="#apiorgapachetikaparserpdfPDFParser.html">PDFParser</a> class parsers Portable Document Format (PDF) documents using the <a class="externalLink" href="http://pdfbox.apache.org/">Apache PDFBox</a> library.</p></div><div class="section"><h3><a name="Electronic_Publication_Format">Electronic Publication Format</a></h3><p>The <a href="#apiorgapach
etikaparserepubEpubParser.html">EpubParser</a> class supports the Electronic Publication Format (EPUB) used for many digital books.</p></div><div class="section"><h3><a name="Rich_Text_Format">Rich Text Format</a></h3><p>The <a href="#apiorgapachetikaparserrtfRTFParser.html">RTFParser</a> class uses the standard javax.swing.text.rtf feature to extract text content from Rich Text Format (RTF) documents.</p></div><div class="section"><h3><a name="Compression_and_packaging_formats">Compression and packaging formats</a></h3><p>Tika uses the <a class="externalLink" href="http://commons.apache.org/compress/">Commons Compress</a> library to support various compression and packaging formats. The <a href="#apiorgapachetikaparserpkgPackageParser.html">PackageParser</a> class and its subclasses first parse the top level compression or packaging format and then pass the unpacked document streams to a second parsing stage using the parser instance specified in the parse context.</p></div><div cl
ass="section"><h3><a name="Text_formats">Text formats</a></h3><p>Extracting text content from plain text files seems like a simple task until you start thinking of all the possible character encodings. The <a href="#apiorgapachetikaparsertxtTXTParser.html">TXTParser</a> class uses encoding detection code from the <a class="externalLink" href="http://site.icu-project.org/">ICU</a> project to automatically detect the character encoding of a text document.</p></div><div class="section"><h3><a name="Audio_formats">Audio formats</a></h3><p>Tika can detect several common audio formats and extract metadata from them. Even text extraction is supported for some audio files that contain lyrics or other textual content. The <a href="#apiorgapachetikaparseraudioAudioParser.html">AudioParser</a> and <a href="#apiorgapachetikaparseraudioMidiParser.html">MidiParser</a> classes use standard javax.sound features to process simple audio formats, and the <a href="#apiorgapachetikaparsermp3Mp3Parser.ht
ml">Mp3Parser</a> class adds support for the widely used MP3 format.</p></div><div class="section"><h3><a name="Image_formats">Image formats</a></h3><p>The <a href="#apiorgapachetikaparserimageImageParser.html">ImageParser</a> class uses the standard javax.imageio feature to extract simple metadata from image formats supported by the Java platform. More complex image metadata is available through the <a href="#apiorgapachetikaparserjpegJpegParser.html">JpegParser</a> class that uses the metadata-extractor library to supports Exif metadata extraction from Jpeg images.</p></div><div class="section"><h3><a name="Video_formats">Video formats</a></h3><p>Currently Tika only supports the Flash video format using a simple parsing algorithm implemented in the <a href="#apiorgapachetikaparserflvFLVParser">FLVParser</a> class.</p></div><div class="section"><h3><a name="Java_class_files_and_archives">Java class files and archives</a></h3><p>The <a href="#apiorgapachetikaparserasmClassParser">Cl
assParser</a> class extracts class names and method signatures from Java class files, and the <a href="#apiorgapachetikaparserpkgZipParser.html">ZipParser</a> class supports also jar archives.</p></div><div class="section"><h3><a name="The_mbox_format">The mbox format</a></h3><p>The <a href="#apiorgapachetikaparsermboxMboxParser.html">MboxParser</a> can extract email messages from the mbox format used by many email archives and Unix-style mailboxes.</p></div></div>
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section">
+<h2>Supported Document Formats<a name="Supported_Document_Formats"></a></h2>
+<p>This page lists all the document formats supported by Apache Tika 0.6. Follow the links to the various parser class javadocs for more detailed information about each document format and how it is parsed by Tika.</p>
+<ul>
+<li><a href="#Supported_Document_Formats">Supported Document Formats</a>
+<ul>
+<li><a href="#HyperText_Markup_Language">HyperText Markup Language</a></li>
+<li><a href="#XML_and_derived_formats">XML and derived formats</a></li>
+<li><a href="#Microsoft_Office_document_formats">Microsoft Office document formats</a></li>
+<li><a href="#OpenDocument_Format">OpenDocument Format</a></li>
+<li><a href="#Portable_Document_Format">Portable Document Format</a></li>
+<li><a href="#Electronic_Publication_Format">Electronic Publication Format</a></li>
+<li><a href="#Rich_Text_Format">Rich Text Format</a></li>
+<li><a href="#Compression_and_packaging_formats">Compression and packaging formats</a></li>
+<li><a href="#Text_formats">Text formats</a></li>
+<li><a href="#Audio_formats">Audio formats</a></li>
+<li><a href="#Image_formats">Image formats</a></li>
+<li><a href="#Video_formats">Video formats</a></li>
+<li><a href="#Java_class_files_and_archives">Java class files and archives</a></li>
+<li><a href="#The_mbox_format">The mbox format</a></li></ul></li></ul>
+<div class="section">
+<h3><a name="HyperText_Markup_Language">HyperText Markup Language</a></h3>
+<p>The HyperText Markup Language (HTML) is the lingua franca of the web. Tika uses the <a class="externalLink" href="http://home.ccil.org/~cowan/XML/tagsoup/">TagSoup</a> library to support virtually any kind of HTML found on the web. The output from the <a href="#apiorgapachetikaparserhtmlHtmlParser.html">HtmlParser</a> class is guaranteed to be well-formed and valid XHTML, and various heuristics are used to prevent things like inline scripts from cluttering the extracted text content.</p></div>
+<div class="section">
+<h3><a name="XML_and_derived_formats">XML and derived formats</a></h3>
+<p>The Extensible Markup Language (XML) format is a generic format that can be used for all kinds of content. Tika has custom parsers for some widely used XML vocabularies like XHTML, OOXML and ODF, but the default <a href="#apiorgapachetikaparserxmlDcXMLParser.html">DcXMLParser</a> class simply extracts the text content of the document and ignores any XML structure. The only exception to this rule are Dublin Core metadata elements that are used for the document metadata.</p></div>
+<div class="section">
+<h3><a name="Microsoft_Office_document_formats">Microsoft Office document formats</a></h3>
+<p>Microsoft Office and some related applications produce documents in the generic OLE 2 Compound Document and Office Open XML (OOXML) formats. The older OLE 2 format was introduced in Microsoft Office version 97 and was the default format until Office version 2007 and the new XML-based OOXML format. The <a href="#apiorgapachetikaparsermicrosoftOfficeParser.html">OfficeParser</a> and <a href="#apiorgapachetikaparsermicrosoftooxmlOOXMLParser.html">OOXMLParser</a> classes use <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> libraries to support text and metadata extraction from both OLE2 and OOXML documents.</p></div>
+<div class="section">
+<h3><a name="OpenDocument_Format">OpenDocument Format</a></h3>
+<p>The OpenDocument format (ODF) is used most notably as the default format of the OpenOffice.org office suite. The <a href="#apiorgapachetikaparserodfOpenDocumentParser.html">OpenDocumentParser</a> class supports this format and the earlier OpenOffice 1.0 format on which ODF is based.</p></div>
+<div class="section">
+<h3><a name="Portable_Document_Format">Portable Document Format</a></h3>
+<p>The <a href="#apiorgapachetikaparserpdfPDFParser.html">PDFParser</a> class parsers Portable Document Format (PDF) documents using the <a class="externalLink" href="http://pdfbox.apache.org/">Apache PDFBox</a> library.</p></div>
+<div class="section">
+<h3><a name="Electronic_Publication_Format">Electronic Publication Format</a></h3>
+<p>The <a href="#apiorgapachetikaparserepubEpubParser.html">EpubParser</a> class supports the Electronic Publication Format (EPUB) used for many digital books.</p></div>
+<div class="section">
+<h3><a name="Rich_Text_Format">Rich Text Format</a></h3>
+<p>The <a href="#apiorgapachetikaparserrtfRTFParser.html">RTFParser</a> class uses the standard javax.swing.text.rtf feature to extract text content from Rich Text Format (RTF) documents.</p></div>
+<div class="section">
+<h3><a name="Compression_and_packaging_formats">Compression and packaging formats</a></h3>
+<p>Tika uses the <a class="externalLink" href="http://commons.apache.org/compress/">Commons Compress</a> library to support various compression and packaging formats. The <a href="#apiorgapachetikaparserpkgPackageParser.html">PackageParser</a> class and its subclasses first parse the top level compression or packaging format and then pass the unpacked document streams to a second parsing stage using the parser instance specified in the parse context.</p></div>
+<div class="section">
+<h3><a name="Text_formats">Text formats</a></h3>
+<p>Extracting text content from plain text files seems like a simple task until you start thinking of all the possible character encodings. The <a href="#apiorgapachetikaparsertxtTXTParser.html">TXTParser</a> class uses encoding detection code from the <a class="externalLink" href="http://site.icu-project.org/">ICU</a> project to automatically detect the character encoding of a text document.</p></div>
+<div class="section">
+<h3><a name="Audio_formats">Audio formats</a></h3>
+<p>Tika can detect several common audio formats and extract metadata from them. Even text extraction is supported for some audio files that contain lyrics or other textual content. The <a href="#apiorgapachetikaparseraudioAudioParser.html">AudioParser</a> and <a href="#apiorgapachetikaparseraudioMidiParser.html">MidiParser</a> classes use standard javax.sound features to process simple audio formats, and the <a href="#apiorgapachetikaparsermp3Mp3Parser.html">Mp3Parser</a> class adds support for the widely used MP3 format.</p></div>
+<div class="section">
+<h3><a name="Image_formats">Image formats</a></h3>
+<p>The <a href="#apiorgapachetikaparserimageImageParser.html">ImageParser</a> class uses the standard javax.imageio feature to extract simple metadata from image formats supported by the Java platform. More complex image metadata is available through the <a href="#apiorgapachetikaparserjpegJpegParser.html">JpegParser</a> class that uses the metadata-extractor library to supports Exif metadata extraction from Jpeg images.</p></div>
+<div class="section">
+<h3><a name="Video_formats">Video formats</a></h3>
+<p>Currently Tika only supports the Flash video format using a simple parsing algorithm implemented in the <a href="#apiorgapachetikaparserflvFLVParser">FLVParser</a> class.</p></div>
+<div class="section">
+<h3><a name="Java_class_files_and_archives">Java class files and archives</a></h3>
+<p>The <a href="#apiorgapachetikaparserasmClassParser">ClassParser</a> class extracts class names and method signatures from Java class files, and the <a href="#apiorgapachetikaparserpkgZipParser.html">ZipParser</a> class supports also jar archives.</p></div>
+<div class="section">
+<h3><a name="The_mbox_format">The mbox format</a></h3>
+<p>The <a href="#apiorgapachetikaparsermboxMboxParser.html">MboxParser</a> can extract email messages from the mbox format used by many email archives and Unix-style mailboxes.</p></div></div>
</div>
<div id="sidebar">
<div id="navigation">
Modified: tika/site/publish/1.3/gettingstarted.html
URL: http://svn.apache.org/viewvc/tika/site/publish/1.3/gettingstarted.html?rev=1582236&r1=1582235&r2=1582236&view=diff
==============================================================================
--- tika/site/publish/1.3/gettingstarted.html (original)
+++ tika/site/publish/1.3/gettingstarted.html Thu Mar 27 09:46:52 2014
@@ -45,7 +45,7 @@
}
}
if (provider == "lucid") {
- form.action = "http://search.lucidimagination.com/p:tika";
+ form.action = "http://find.searchhub.org/p:tika";
} else if (provider == "sl") {
form.action = "http://search-lucene.com/tika";
}
@@ -84,15 +84,53 @@
width="387" height="100"/></a>
</div>
<div id="content">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Getting Started with Apache Tika<a name="Getting_Started_with_Apache_Tika"></a></h2><p>This document
describes how to build Apache Tika from sources and how to start using Tika in an application.</p></div><div class="section"><h2>Getting and building the sources<a name="Getting_and_building_the_sources"></a></h2><p>To build Tika from sources you first need to either <a href="../download.html">download</a> a source release or <a href="../source-repository.html">checkout</a> the latest sources from version control.</p><p>Once you have the sources, you can build them using the <a class="externalLink" href="http://maven.apache.org/">Maven 2</a> build system. Executing the following command in the base directory will build the sources and install the resulting artifacts in your local Maven repository.</p><div><pre>mvn install</pre></div><p>See the Maven documentation for more information about the available build options.</p><p>Note that you need Java 5 or higher to build Tika.</p></div><div class="section"><h2>Build artifacts<a name="Build_artifacts"></a></h2><p>The Tika build consists
of a number of components and produces the following main binaries:</p><dl><dt>tika-core/target/tika-core-*.jar</dt><dd> Tika core library. Contains the core interfaces and classes of Tika, but none of the parser implementations. Depends only on Java 5.</dd><dt>tika-parsers/target/tika-parsers-*.jar</dt><dd> Tika parsers. Collection of classes that implement the Tika Parser interface based on various external parser libraries.</dd><dt>tika-app/target/tika-app-*.jar</dt><dd> Tika application. Combines the above components and all the external parser libraries into a single runnable jar with a GUI and a command line interface.</dd><dt>tika-bundle/target/tika-bundle-*.jar</dt><dd> Tika bundle. An OSGi bundle that combines tika-parsers with non-OSGified parser libraries to make them easy to deploy in an OSGi environment.</dd></dl></div><div class="section"><h2>Using Tika as a Maven dependency<a name="Using_Tika_as_a_Maven_dependency"></a></h2><p>The core library, tika-core, contains th
e key interfaces and classes of Tika and can be used by itself if you don't need the full set of parsers from the tika-parsers component. The tika-core dependency looks like this:</p><div><pre> <dependency>
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section">
+<h2>Getting Started with Apache Tika<a name="Getting_Started_with_Apache_Tika"></a></h2>
+<p>This document describes how to build Apache Tika from sources and how to start using Tika in an application.</p></div>
+<div class="section">
+<h2>Getting and building the sources<a name="Getting_and_building_the_sources"></a></h2>
+<p>To build Tika from sources you first need to either <a href="../download.html">download</a> a source release or <a href="../source-repository.html">checkout</a> the latest sources from version control.</p>
+<p>Once you have the sources, you can build them using the <a class="externalLink" href="http://maven.apache.org/">Maven 2</a> build system. Executing the following command in the base directory will build the sources and install the resulting artifacts in your local Maven repository.</p>
+<div>
+<pre>mvn install</pre></div>
+<p>See the Maven documentation for more information about the available build options.</p>
+<p>Note that you need Java 5 or higher to build Tika.</p></div>
+<div class="section">
+<h2>Build artifacts<a name="Build_artifacts"></a></h2>
+<p>The Tika build consists of a number of components and produces the following main binaries:</p>
+<dl>
+<dt>tika-core/target/tika-core-*.jar</dt>
+<dd> Tika core library. Contains the core interfaces and classes of Tika, but none of the parser implementations. Depends only on Java 5.</dd>
+<dt>tika-parsers/target/tika-parsers-*.jar</dt>
+<dd> Tika parsers. Collection of classes that implement the Tika Parser interface based on various external parser libraries.</dd>
+<dt>tika-app/target/tika-app-*.jar</dt>
+<dd> Tika application. Combines the above components and all the external parser libraries into a single runnable jar with a GUI and a command line interface.</dd>
+<dt>tika-bundle/target/tika-bundle-*.jar</dt>
+<dd> Tika bundle. An OSGi bundle that combines tika-parsers with non-OSGified parser libraries to make them easy to deploy in an OSGi environment.</dd></dl></div>
+<div class="section">
+<h2>Using Tika as a Maven dependency<a name="Using_Tika_as_a_Maven_dependency"></a></h2>
+<p>The core library, tika-core, contains the key interfaces and classes of Tika and can be used by itself if you don't need the full set of parsers from the tika-parsers component. The tika-core dependency looks like this:</p>
+<div>
+<pre> <dependency>
<groupId>org.apache.tika</groupId>
<artifactId>tika-core</artifactId>
<version>...</version>
- </dependency></pre></div><p>If you want to use Tika to parse documents (instead of simply detecting document types, etc.), you'll want to depend on tika-parsers instead: </p><div><pre> <dependency>
+ </dependency></pre></div>
+<p>If you want to use Tika to parse documents (instead of simply detecting document types, etc.), you'll want to depend on tika-parsers instead: </p>
+<div>
+<pre> <dependency>
<groupId>org.apache.tika</groupId>
<artifactId>tika-parsers</artifactId>
<version>...</version>
- </dependency></pre></div><p>Note that adding this dependency will introduce a number of transitive dependencies to your project, including one on tika-core. You need to make sure that these dependencies won't conflict with your existing project dependencies. You can use the following command in the tika-parsers directory to get a full listing of all the dependencies.</p><div><pre>$ mvn dependency:tree | grep :compile</pre></div></div><div class="section"><h2>Using Tika in an Ant project<a name="Using_Tika_in_an_Ant_project"></a></h2><p>Unless you use a dependency manager tool like <a class="externalLink" href="http://ant.apache.org/ivy/">Apache Ivy</a>, the easiest way to use Tika is to include either the tika-core or the tika-app jar in your classpath, depending on whether you want just the core functionality or also all the parser implementations.</p><div><pre><classpath>
+ </dependency></pre></div>
+<p>Note that adding this dependency will introduce a number of transitive dependencies to your project, including one on tika-core. You need to make sure that these dependencies won't conflict with your existing project dependencies. You can use the following command in the tika-parsers directory to get a full listing of all the dependencies.</p>
+<div>
+<pre>$ mvn dependency:tree | grep :compile</pre></div></div>
+<div class="section">
+<h2>Using Tika in an Ant project<a name="Using_Tika_in_an_Ant_project"></a></h2>
+<p>Unless you use a dependency manager tool like <a class="externalLink" href="http://ant.apache.org/ivy/">Apache Ivy</a>, the easiest way to use Tika is to include either the tika-core or the tika-app jar in your classpath, depending on whether you want just the core functionality or also all the parser implementations.</p>
+<div>
+<pre><classpath>
... <!-- your other classpath entries -->
<!-- either: -->
@@ -100,7 +138,13 @@
<!-- or: -->
<pathelement location="path/to/tika-app-${tika.version}.jar"/>
-</classpath></pre></div></div><div class="section"><h2>Using Tika as a command line utility<a name="Using_Tika_as_a_command_line_utility"></a></h2><p>The Tika application jar (tika-app-*.jar) can be used as a command line utility for extracting text content and metadata from all sorts of files. This runnable jar contains all the dependencies it needs, so you don't need to worry about classpath settings to run it.</p><p>The usage instructions are shown below.</p><div><pre>usage: java -jar tika-app.jar [option...] [file|port...]
+</classpath></pre></div></div>
+<div class="section">
+<h2>Using Tika as a command line utility<a name="Using_Tika_as_a_command_line_utility"></a></h2>
+<p>The Tika application jar (tika-app-*.jar) can be used as a command line utility for extracting text content and metadata from all sorts of files. This runnable jar contains all the dependencies it needs, so you don't need to worry about classpath settings to run it.</p>
+<p>The usage instructions are shown below.</p>
+<div>
+<pre>usage: java -jar tika-app.jar [option...] [file|port...]
Options:
-? or --help Print this usage message
@@ -164,7 +208,10 @@ Description:
Use the "--server" (or "-s") option to start the
Apache Tika server. The server will listen to the
- ports you specify as one or more arguments.</pre></div><p>You can also use the jar as a component in a Unix pipeline or as an external tool in many scripting languages.</p><div><pre># Check if an Internet resource contains a specific keyword
+ ports you specify as one or more arguments.</pre></div>
+<p>You can also use the jar as a component in a Unix pipeline or as an external tool in many scripting languages.</p>
+<div>
+<pre># Check if an Internet resource contains a specific keyword
curl http://.../document.doc \
| java -jar tika-app.jar --text \
| grep -q keyword</pre></div></div>
Modified: tika/site/publish/1.3/index.html
URL: http://svn.apache.org/viewvc/tika/site/publish/1.3/index.html?rev=1582236&r1=1582235&r2=1582236&view=diff
==============================================================================
--- tika/site/publish/1.3/index.html (original)
+++ tika/site/publish/1.3/index.html Thu Mar 27 09:46:52 2014
@@ -45,7 +45,7 @@
}
}
if (provider == "lucid") {
- form.action = "http://search.lucidimagination.com/p:tika";
+ form.action = "http://find.searchhub.org/p:tika";
} else if (provider == "sl") {
form.action = "http://search-lucene.com/tika";
}
@@ -84,7 +84,56 @@
width="387" height="100"/></a>
</div>
<div id="content">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Apache Tika 1.3<a name="Apache_Tika_1.3"></a></h2><p>The most notable changes in Tika 1.3 over the pr
evious release are:</p><ul><li>Mimetype definitions added for more common programming languages, including common extensions, but not magic patterns. (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1055">TIKA-1055</a>)</li><li>MS Word: When a Word (.doc) document contains embedded files or links to external documents, Tika now places a div class="embedded" id="_XXX"/ placeholder into the XHTML so you can see where in the main text the embedded document occurred (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-956">TIKA-956</a>, <a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1019">TIKA-1019</a>). </li><li>Embedded Wordpad/RTF documents are now recognized (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-982">TIKA-982</a>).</li><li>PDF: Text from pop-up annotations is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-981">TIKA-981</a
>). Text from bookmarks is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1035">TIKA-1035</a>).</li><li>PKCS7: Detached signatures no longer through NullPointerException (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-986">TIKA-986</a>).</li><li>iWork: The chart name for charts embedded in numbers documents is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-918">TIKA-918</a>).</li><li>CLI: TikaCLI -m now handles multi-valued metadata keys correctly (previously it only printed the first value). (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-920">TIKA-920</a>)</li><li>MS Word (.docx): When a Word (.docx) document contains embedded files, Tika now places a div class="embedded" id="XXX"/ into the XHTML so you can see where in the main text the embedded document occurred. The id (rId) is included in the Metadata of each embedded document as
the new Metadata.EMBEDDED_RELATIONSHIP_ID key, and TikaCLI prepends the rId (if present) onto the filename it extracts (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-989">TIKA-989</a>). Fixed NullPointerException when style is null (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1006">TIKA-1006</a>). Text inside text boxes is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1005">TIKA-1005</a>).</li><li>RTF: Page, word, character count and creation date metadata are now extracted for RTF documents (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-999">TIKA-999</a>). MS PowerPoint (.pptx): When a PowerPoint (.pptx) document contains embedded files, Tika now places a div class="embedded" id="XXX"/ into the XHTML so you can see where in the main text the embedded document occurred. The id (rId) is included in the Metadata of each embedded document as the
new Metadata.EMBEDDED_RELATIONSHIP_ID key, and TikaCLI prepends the rId (if present) onto the filename it extracts (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-997">TIKA-997</a>, <a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1032">TIKA-1032</a>). MS PowerPoint (.ppt): When a PowerPoint (.ppt) document contains embedded files, Tika now places a div class="embedded" id="XXX"/ into the XHTML so you can see where in the main text the embedded document occurred (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA--1025">TIKA-1025</a>). Text from the master slide is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-712">TIKA-712</a>).</li><li>MHTML: fixed Null charset name exception when a mime part has an unrecognized charset (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1011">TIKA-1011</a>).</li><li>MP3: if an ID3 tag was encoded i
n UTF-16 with only the BOM then on certain JVMs this would incorrectly extract the BOM as the tag's value (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1024">TIKA-1024</a>).</li><li>ZIP: placeholders (div class="embedded" id="<i>entry name"/</i>) are now left in the XHTML so you can see where each archive member appears (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1036">TIKA-1036</a>). TikaCLI would hit FileNotFoundException when extracting files that were under sub-directories from a ZIP archive, because it failed to create the parent directories first (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1031">TIKA-1031</a>).</li><li>XML: a space character is now added before each element (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1048">TIKA-1048</a>)</li></ul><p>The following people have contributed to Tika 1.3 by submitting or commenting on the issues resol
ved in this release:</p><ul><li>Andrew Jackson</li><li>Arthur Meneau</li><li>Benoit MAGGI</li><li>Bernhard Berger</li><li>Chris A. Mattmann</li><li>Christoph Brill</li><li>Daniel Bonniot de Ruisselet</li><li>David A. Patterson</li><li>David Morana</li><li>Emmanuel Hugonnet</li><li>Erik Peterson</li><li>Gary Karasiuk</li><li>John Conwell</li><li>Jonas Wilhelmsson</li><li>Jukka Zitting</li><li>Karel Zacek</li><li>Ken Krugler</li><li>Maciej Lizewski</li><li>Marco Quaranta</li><li>Markus Jelsma</li><li>Michael McCandless</li><li>Nick Burch</li><li>Oliver Heger</li><li>Paolo Nacci</li><li>Qian Diao</li><li>Ray Gauss II</li><li>Richard Eccles</li><li>Ryan McKinley</li><li>Shinichiro Abe</li><li>Sture Svensson</li></ul><p>See <a class="externalLink" href="http://s.apache.org/lYv">http://s.apache.org/lYv</a> for more details on these contributions.</p></div>
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section">
+<h2>Apache Tika 1.3<a name="Apache_Tika_1.3"></a></h2>
+<p>The most notable changes in Tika 1.3 over the previous release are:</p>
+<ul>
+<li>Mimetype definitions added for more common programming languages, including common extensions, but not magic patterns. (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1055">TIKA-1055</a>)</li>
+<li>MS Word: When a Word (.doc) document contains embedded files or links to external documents, Tika now places a div class="embedded" id="_XXX"/ placeholder into the XHTML so you can see where in the main text the embedded document occurred (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-956">TIKA-956</a>, <a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1019">TIKA-1019</a>). </li>
+<li>Embedded Wordpad/RTF documents are now recognized (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-982">TIKA-982</a>).</li>
+<li>PDF: Text from pop-up annotations is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-981">TIKA-981</a>). Text from bookmarks is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1035">TIKA-1035</a>).</li>
+<li>PKCS7: Detached signatures no longer through NullPointerException (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-986">TIKA-986</a>).</li>
+<li>iWork: The chart name for charts embedded in numbers documents is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-918">TIKA-918</a>).</li>
+<li>CLI: TikaCLI -m now handles multi-valued metadata keys correctly (previously it only printed the first value). (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-920">TIKA-920</a>)</li>
+<li>MS Word (.docx): When a Word (.docx) document contains embedded files, Tika now places a div class="embedded" id="XXX"/ into the XHTML so you can see where in the main text the embedded document occurred. The id (rId) is included in the Metadata of each embedded document as the new Metadata.EMBEDDED_RELATIONSHIP_ID key, and TikaCLI prepends the rId (if present) onto the filename it extracts (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-989">TIKA-989</a>). Fixed NullPointerException when style is null (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1006">TIKA-1006</a>). Text inside text boxes is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1005">TIKA-1005</a>).</li>
+<li>RTF: Page, word, character count and creation date metadata are now extracted for RTF documents (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-999">TIKA-999</a>). MS PowerPoint (.pptx): When a PowerPoint (.pptx) document contains embedded files, Tika now places a div class="embedded" id="XXX"/ into the XHTML so you can see where in the main text the embedded document occurred. The id (rId) is included in the Metadata of each embedded document as the new Metadata.EMBEDDED_RELATIONSHIP_ID key, and TikaCLI prepends the rId (if present) onto the filename it extracts (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-997">TIKA-997</a>, <a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1032">TIKA-1032</a>). MS PowerPoint (.ppt): When a PowerPoint (.ppt) document contains embedded files, Tika now places a div class="embedded" id="XXX"/ into the XHTML so you can see where in the
main text the embedded document occurred (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA--1025">TIKA-1025</a>). Text from the master slide is now extracted (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-712">TIKA-712</a>).</li>
+<li>MHTML: fixed Null charset name exception when a mime part has an unrecognized charset (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1011">TIKA-1011</a>).</li>
+<li>MP3: if an ID3 tag was encoded in UTF-16 with only the BOM then on certain JVMs this would incorrectly extract the BOM as the tag's value (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1024">TIKA-1024</a>).</li>
+<li>ZIP: placeholders (div class="embedded" id="<i>entry name"/</i>) are now left in the XHTML so you can see where each archive member appears (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1036">TIKA-1036</a>). TikaCLI would hit FileNotFoundException when extracting files that were under sub-directories from a ZIP archive, because it failed to create the parent directories first (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1031">TIKA-1031</a>).</li>
+<li>XML: a space character is now added before each element (<a class="externalLink" href="http://issues.apache.org/jira/browse/TIKA-1048">TIKA-1048</a>)</li></ul>
+<p>The following people have contributed to Tika 1.3 by submitting or commenting on the issues resolved in this release:</p>
+<ul>
+<li>Andrew Jackson</li>
+<li>Arthur Meneau</li>
+<li>Benoit MAGGI</li>
+<li>Bernhard Berger</li>
+<li>Chris A. Mattmann</li>
+<li>Christoph Brill</li>
+<li>Daniel Bonniot de Ruisselet</li>
+<li>David A. Patterson</li>
+<li>David Morana</li>
+<li>Emmanuel Hugonnet</li>
+<li>Erik Peterson</li>
+<li>Gary Karasiuk</li>
+<li>John Conwell</li>
+<li>Jonas Wilhelmsson</li>
+<li>Jukka Zitting</li>
+<li>Karel Zacek</li>
+<li>Ken Krugler</li>
+<li>Maciej Lizewski</li>
+<li>Marco Quaranta</li>
+<li>Markus Jelsma</li>
+<li>Michael McCandless</li>
+<li>Nick Burch</li>
+<li>Oliver Heger</li>
+<li>Paolo Nacci</li>
+<li>Qian Diao</li>
+<li>Ray Gauss II</li>
+<li>Richard Eccles</li>
+<li>Ryan McKinley</li>
+<li>Shinichiro Abe</li>
+<li>Sture Svensson</li></ul>
+<p>See <a class="externalLink" href="http://s.apache.org/lYv">http://s.apache.org/lYv</a> for more details on these contributions.</p></div>
</div>
<div id="sidebar">
<div id="navigation">
Modified: tika/site/publish/1.3/parser.html
URL: http://svn.apache.org/viewvc/tika/site/publish/1.3/parser.html?rev=1582236&r1=1582235&r2=1582236&view=diff
==============================================================================
--- tika/site/publish/1.3/parser.html (original)
+++ tika/site/publish/1.3/parser.html Thu Mar 27 09:46:52 2014
@@ -45,7 +45,7 @@
}
}
if (provider == "lucid") {
- form.action = "http://search.lucidimagination.com/p:tika";
+ form.action = "http://find.searchhub.org/p:tika";
} else if (provider == "sl") {
form.action = "http://search-lucene.com/tika";
}
@@ -84,28 +84,96 @@
width="387" height="100"/></a>
</div>
<div id="content">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>The Parser interface<a name="The_Parser_interface"></a></h2><p>The <a href="#apiorgapachetikaparserPa
rser.html">org.apache.tika.parser.Parser</a> interface is the key concept of Apache Tika. It hides the complexity of different file formats and parsing libraries while providing a simple and powerful mechanism for client applications to extract structured text content and metadata from all sorts of documents. All this is achieved with a single method:</p><div><pre>void parse(
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section">
+<h2>The Parser interface<a name="The_Parser_interface"></a></h2>
+<p>The <a href="#apiorgapachetikaparserParser.html">org.apache.tika.parser.Parser</a> interface is the key concept of Apache Tika. It hides the complexity of different file formats and parsing libraries while providing a simple and powerful mechanism for client applications to extract structured text content and metadata from all sorts of documents. All this is achieved with a single method:</p>
+<div>
+<pre>void parse(
InputStream stream, ContentHandler handler, Metadata metadata,
- ParseContext context) throws IOException, SAXException, TikaException;</pre></div><p>The <tt>parse</tt> method takes the document to be parsed and related metadata as input and outputs the results as XHTML SAX events and extra metadata. The parse context argument is used to specify context information (like the current local) that is not related to any individual document. The main criteria that lead to this design were:</p><dl><dt>Streamed parsing</dt><dd>The interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. This allows even huge documents to be parsed without excessive resource requirements.</dd><dt>Structured content</dt><dd>A parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. A client application can use this information for example to better judge the relevance of different parts of the parsed document.</dd
><dt>Input metadata</dt><dd>A client application should be able to include metadata like the file name or declared content type with the document to be parsed. The parser implementation can use this information to better guide the parsing process.</dd><dt>Output metadata</dt><dd>A parser implementation should be able to return document metadata in addition to document content. Many document formats contain metadata like the name of the author that may be useful to client applications.</dd><dt>Context sensitivity</dt><dd>While the default settings and behaviour of Tika parsers should work well for most use cases, there are still situations where more fine-grained control over the parsing process is desirable. It should be easy to inject such context-specific information to the parsing process without breaking the layers of abstraction.</dd></dl><p>These criteria are reflected in the arguments of the <tt>parse</tt> method.</p><div class="section"><h3>Document input stream<a name="Docu
ment_input_stream"></a></h3><p>The first argument is an <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html">InputStream</a> for reading the document to be parsed.</p><p>If this document stream can not be read, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html">IOException</a> is passed up to the client application. If the stream can be read but not parsed (for example if the document is corrupted), then the parser throws a <a href="#apiorgapachetikaexceptionTikaException.html">TikaException</a>.</p><p>The parser implementation will consume this stream but <i>will not close it</i>. Closing the stream is the responsibility of the client application that opened it in the first place. The recommended pattern for using streams with the <tt>parse</tt> method is:</p><div><pre>InputStream stream = ...; // open the stream
+ ParseContext context) throws IOException, SAXException, TikaException;</pre></div>
+<p>The <tt>parse</tt> method takes the document to be parsed and related metadata as input and outputs the results as XHTML SAX events and extra metadata. The parse context argument is used to specify context information (like the current local) that is not related to any individual document. The main criteria that lead to this design were:</p>
+<dl>
+<dt>Streamed parsing</dt>
+<dd>The interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. This allows even huge documents to be parsed without excessive resource requirements.</dd>
+<dt>Structured content</dt>
+<dd>A parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. A client application can use this information for example to better judge the relevance of different parts of the parsed document.</dd>
+<dt>Input metadata</dt>
+<dd>A client application should be able to include metadata like the file name or declared content type with the document to be parsed. The parser implementation can use this information to better guide the parsing process.</dd>
+<dt>Output metadata</dt>
+<dd>A parser implementation should be able to return document metadata in addition to document content. Many document formats contain metadata like the name of the author that may be useful to client applications.</dd>
+<dt>Context sensitivity</dt>
+<dd>While the default settings and behaviour of Tika parsers should work well for most use cases, there are still situations where more fine-grained control over the parsing process is desirable. It should be easy to inject such context-specific information to the parsing process without breaking the layers of abstraction.</dd></dl>
+<p>These criteria are reflected in the arguments of the <tt>parse</tt> method.</p>
+<div class="section">
+<h3>Document input stream<a name="Document_input_stream"></a></h3>
+<p>The first argument is an <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html">InputStream</a> for reading the document to be parsed.</p>
+<p>If this document stream can not be read, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html">IOException</a> is passed up to the client application. If the stream can be read but not parsed (for example if the document is corrupted), then the parser throws a <a href="#apiorgapachetikaexceptionTikaException.html">TikaException</a>.</p>
+<p>The parser implementation will consume this stream but <i>will not close it</i>. Closing the stream is the responsibility of the client application that opened it in the first place. The recommended pattern for using streams with the <tt>parse</tt> method is:</p>
+<div>
+<pre>InputStream stream = ...; // open the stream
try {
parser.parse(stream, ...); // parse the stream
} finally {
stream.close(); // close the stream
-}</pre></div><p>Some document formats like the OLE2 Compound Document Format used by Microsoft Office are best parsed as random access files. In such cases the content of the input stream is automatically spooled to a temporary file that gets removed once parsed. A future version of Tika may make it possible to avoid this extra file if the input document is already a file in the local file system. See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-153">TIKA-153</a> for the status of this feature request.</p></div><div class="section"><h3>XHTML SAX events<a name="XHTML_SAX_events"></a></h3><p>The parsed content of the document stream is returned to the client application as a sequence of XHTML SAX events. XHTML is used to express structured content of the document and SAX events enable streamed processing. Note that the XHTML format is used here only to convey structural information, not to render the documents for browsing!</p><p>The XHTML SAX events produc
ed by the parser implementation are sent to a <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/org/xml/sax/ContentHandler.html">ContentHandler</a> instance given to the <tt>parse</tt> method. If this the content handler fails to process an event, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/org/xml/sax/SAXException.html">SAXException</a> is passed up to the client application.</p><p>The overall structure of the generated event stream is (with indenting added for clarity):</p><div><pre><html xmlns="http://www.w3.org/1999/xhtml">
+}</pre></div>
+<p>Some document formats like the OLE2 Compound Document Format used by Microsoft Office are best parsed as random access files. In such cases the content of the input stream is automatically spooled to a temporary file that gets removed once parsed. A future version of Tika may make it possible to avoid this extra file if the input document is already a file in the local file system. See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-153">TIKA-153</a> for the status of this feature request.</p></div>
+<div class="section">
+<h3>XHTML SAX events<a name="XHTML_SAX_events"></a></h3>
+<p>The parsed content of the document stream is returned to the client application as a sequence of XHTML SAX events. XHTML is used to express structured content of the document and SAX events enable streamed processing. Note that the XHTML format is used here only to convey structural information, not to render the documents for browsing!</p>
+<p>The XHTML SAX events produced by the parser implementation are sent to a <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/org/xml/sax/ContentHandler.html">ContentHandler</a> instance given to the <tt>parse</tt> method. If this the content handler fails to process an event, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/org/xml/sax/SAXException.html">SAXException</a> is passed up to the client application.</p>
+<p>The overall structure of the generated event stream is (with indenting added for clarity):</p>
+<div>
+<pre><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>...</title>
</head>
<body>
...
</body>
-</html></pre></div><p>Parser implementations typically use the <a href="#apidocsorgapachetikasaxXHTMLContentHandler.html">XHTMLContentHandler</a> utility class to generate the XHTML output.</p><p>Dealing with the raw SAX events can be a bit complex, so Apache Tika comes with a number of utility classes that can be used to process and convert the event stream to other representations.</p><p>For example, the <a href="#apiorgapachetikasaxBodyContentHandler.html">BodyContentHandler</a> class can be used to extract just the body part of the XHTML output and feed it either as SAX events to another content handler or as characters to an output stream, a writer, or simply a string. The following code snippet parses a document from the standard input stream and outputs the extracted text content to standard output:</p><div><pre>ContentHandler handler = new BodyContentHandler(System.out);
-parser.parse(System.in, handler, ...);</pre></div><p>Another useful class is <a href="#apiorgapachetikaparserParsingReader.html">ParsingReader</a> that uses a background thread to parse the document and returns the extracted text content as a character stream:</p><div><pre>InputStream stream = ...; // the document to be parsed
+</html></pre></div>
+<p>Parser implementations typically use the <a href="#apidocsorgapachetikasaxXHTMLContentHandler.html">XHTMLContentHandler</a> utility class to generate the XHTML output.</p>
+<p>Dealing with the raw SAX events can be a bit complex, so Apache Tika comes with a number of utility classes that can be used to process and convert the event stream to other representations.</p>
+<p>For example, the <a href="#apiorgapachetikasaxBodyContentHandler.html">BodyContentHandler</a> class can be used to extract just the body part of the XHTML output and feed it either as SAX events to another content handler or as characters to an output stream, a writer, or simply a string. The following code snippet parses a document from the standard input stream and outputs the extracted text content to standard output:</p>
+<div>
+<pre>ContentHandler handler = new BodyContentHandler(System.out);
+parser.parse(System.in, handler, ...);</pre></div>
+<p>Another useful class is <a href="#apiorgapachetikaparserParsingReader.html">ParsingReader</a> that uses a background thread to parse the document and returns the extracted text content as a character stream:</p>
+<div>
+<pre>InputStream stream = ...; // the document to be parsed
Reader reader = new ParsingReader(parser, stream, ...);
try {
...; // read the document text using the reader
} finally {
reader.close(); // the document stream is closed automatically
-}</pre></div></div><div class="section"><h3>Document metadata<a name="Document_metadata"></a></h3><p>The third argument to the <tt>parse</tt> method is used to pass document metadata both in and out of the parser. Document metadata is expressed as an <a href="#apiorgapachetikametadataMetadata.html">Metadata</a> object.</p><p>The following are some of the more interesting metadata properties:</p><dl><dt>Metadata.RESOURCE_NAME_KEY</dt><dd>The name of the file or resource that contains the document.<p>A client application can set this property to allow the parser to use file name heuristics to determine the format of the document.</p><p>The parser implementation may set this property if the file format contains the canonical name of the file (for example the Gzip format has a slot for the file name).</p></dd><dt>Metadata.CONTENT_TYPE</dt><dd>The declared content type of the document.<p>A client application can set this property based on for example a HTTP Content-Type header. The decla
red content type may help the parser to correctly interpret the document.</p><p>The parser implementation sets this property to the content type according to which the document was parsed.</p></dd><dt>Metadata.TITLE</dt><dd>The title of the document.<p>The parser implementation sets this property if the document format contains an explicit title field.</p></dd><dt>Metadata.AUTHOR</dt><dd>The name of the author of the document.<p>The parser implementation sets this property if the document format contains an explicit author field.</p></dd></dl><p>Note that metadata handling is still being discussed by the Tika development team, and it is likely that there will be some (backwards incompatible) changes in metadata handling before Tika 1.0.</p></div><div class="section"><h3>Parse context<a name="Parse_context"></a></h3><p>The final argument to the <tt>parse</tt> method is used to inject context-specific information to the parsing process. This is useful for example when dealing with loc
ale-specific date and number formats in Microsoft Excel spreadsheets. Another important use of the parse context is passing in the delegate parser instance to be used by two-phase parsers like the <a href="#apiorgapacheparserpkgPackageParser.html">PackageParser</a> subclasses. Some parser classes allow customization of the parsing process through strategy objects in the parse context.</p></div><div class="section"><h3>Parser implementations<a name="Parser_implementations"></a></h3><p>Apache Tika comes with a number of parser classes for parsing <a href="#formats.html">various document formats</a>. You can also extend Tika with your own parsers, and of course any contributions to Tika are warmly welcome.</p><p>The goal of Tika is to reuse existing parser libraries like <a class="externalLink" href="http://www.pdfbox.org/">PDFBox</a> or <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> as much as possible, and so most of the parser classes in Tika are adapters to su
ch external libraries.</p><p>Tika also contains some general purpose parser implementations that are not targeted at any specific document formats. The most notable of these is the <a href="#apidocsorgapachetikaparserAutoDetectParser.html">AutoDetectParser</a> class that encapsulates all Tika functionality into a single parser that can handle any types of documents. This parser will automatically determine the type of the incoming document based on various heuristics and will then parse the document accordingly.</p></div></div>
+}</pre></div></div>
+<div class="section">
+<h3>Document metadata<a name="Document_metadata"></a></h3>
+<p>The third argument to the <tt>parse</tt> method is used to pass document metadata both in and out of the parser. Document metadata is expressed as an <a href="#apiorgapachetikametadataMetadata.html">Metadata</a> object.</p>
+<p>The following are some of the more interesting metadata properties:</p>
+<dl>
+<dt>Metadata.RESOURCE_NAME_KEY</dt>
+<dd>The name of the file or resource that contains the document.
+<p>A client application can set this property to allow the parser to use file name heuristics to determine the format of the document.</p>
+<p>The parser implementation may set this property if the file format contains the canonical name of the file (for example the Gzip format has a slot for the file name).</p></dd>
+<dt>Metadata.CONTENT_TYPE</dt>
+<dd>The declared content type of the document.
+<p>A client application can set this property based on for example a HTTP Content-Type header. The declared content type may help the parser to correctly interpret the document.</p>
+<p>The parser implementation sets this property to the content type according to which the document was parsed.</p></dd>
+<dt>Metadata.TITLE</dt>
+<dd>The title of the document.
+<p>The parser implementation sets this property if the document format contains an explicit title field.</p></dd>
+<dt>Metadata.AUTHOR</dt>
+<dd>The name of the author of the document.
+<p>The parser implementation sets this property if the document format contains an explicit author field.</p></dd></dl>
+<p>Note that metadata handling is still being discussed by the Tika development team, and it is likely that there will be some (backwards incompatible) changes in metadata handling before Tika 1.0.</p></div>
+<div class="section">
+<h3>Parse context<a name="Parse_context"></a></h3>
+<p>The final argument to the <tt>parse</tt> method is used to inject context-specific information to the parsing process. This is useful for example when dealing with locale-specific date and number formats in Microsoft Excel spreadsheets. Another important use of the parse context is passing in the delegate parser instance to be used by two-phase parsers like the <a href="#apiorgapacheparserpkgPackageParser.html">PackageParser</a> subclasses. Some parser classes allow customization of the parsing process through strategy objects in the parse context.</p></div>
+<div class="section">
+<h3>Parser implementations<a name="Parser_implementations"></a></h3>
+<p>Apache Tika comes with a number of parser classes for parsing <a href="#formats.html">various document formats</a>. You can also extend Tika with your own parsers, and of course any contributions to Tika are warmly welcome.</p>
+<p>The goal of Tika is to reuse existing parser libraries like <a class="externalLink" href="http://www.pdfbox.org/">PDFBox</a> or <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> as much as possible, and so most of the parser classes in Tika are adapters to such external libraries.</p>
+<p>Tika also contains some general purpose parser implementations that are not targeted at any specific document formats. The most notable of these is the <a href="#apidocsorgapachetikaparserAutoDetectParser.html">AutoDetectParser</a> class that encapsulates all Tika functionality into a single parser that can handle any types of documents. This parser will automatically determine the type of the incoming document based on various heuristics and will then parse the document accordingly.</p></div></div>
</div>
<div id="sidebar">
<div id="navigation">
Modified: tika/site/publish/1.3/parser_guide.html
URL: http://svn.apache.org/viewvc/tika/site/publish/1.3/parser_guide.html?rev=1582236&r1=1582235&r2=1582236&view=diff
==============================================================================
--- tika/site/publish/1.3/parser_guide.html (original)
+++ tika/site/publish/1.3/parser_guide.html Thu Mar 27 09:46:52 2014
@@ -45,7 +45,7 @@
}
}
if (provider == "lucid") {
- form.action = "http://search.lucidimagination.com/p:tika";
+ form.action = "http://find.searchhub.org/p:tika";
} else if (provider == "sl") {
form.action = "http://search-lucene.com/tika";
}
@@ -84,9 +84,31 @@
width="387" height="100"/></a>
</div>
<div id="content">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Get Tika parsing up and running in 5 minutes<a name="Get_Tika_parsing_up_and_running_in_5_minutes"></
a></h2><p>This page is a quick start guide showing how to add a new parser to Apache Tika. Following the simple steps listed below your new parser can be running in only 5 minutes.</p><ul><li><a href="#Get_Tika_parsing_up_and_running_in_5_minutes">Get Tika parsing up and running in 5 minutes</a><ul><li><a href="#Getting_Started">Getting Started</a></li><li><a href="#Add_your_MIME-Type">Add your MIME-Type</a></li><li><a href="#Create_your_Parser_class">Create your Parser class</a></li><li><a href="#List_the_new_parser">List the new parser</a></li></ul></li></ul><div class="section"><h3><a name="Getting_Started">Getting Started</a></h3><p>The <a href="#gettingstarted.html">Getting Started</a> document describes how to build Apache Tika from sources and how to start using Tika in an application. Pay close attention and follow the instructions in the "Getting and building the sources" section.</p></div><div class="section"><h3><a name="Add_your_MIME-Type">Add your MIME-Type</a
></h3><p>You first need to modify <a class="externalLink" href="http://svn.apache.org/repos/asf/tika/trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml">tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml</a> in order to Tika can map the file extension with its MIME-Type. You should add something like this:</p><div><pre> <mime-type type="application/hello">
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section">
+<h2>Get Tika parsing up and running in 5 minutes<a name="Get_Tika_parsing_up_and_running_in_5_minutes"></a></h2>
+<p>This page is a quick start guide showing how to add a new parser to Apache Tika. Following the simple steps listed below your new parser can be running in only 5 minutes.</p>
+<ul>
+<li><a href="#Get_Tika_parsing_up_and_running_in_5_minutes">Get Tika parsing up and running in 5 minutes</a>
+<ul>
+<li><a href="#Getting_Started">Getting Started</a></li>
+<li><a href="#Add_your_MIME-Type">Add your MIME-Type</a></li>
+<li><a href="#Create_your_Parser_class">Create your Parser class</a></li>
+<li><a href="#List_the_new_parser">List the new parser</a></li></ul></li></ul>
+<div class="section">
+<h3><a name="Getting_Started">Getting Started</a></h3>
+<p>The <a href="#gettingstarted.html">Getting Started</a> document describes how to build Apache Tika from sources and how to start using Tika in an application. Pay close attention and follow the instructions in the "Getting and building the sources" section.</p></div>
+<div class="section">
+<h3><a name="Add_your_MIME-Type">Add your MIME-Type</a></h3>
+<p>You first need to modify <a class="externalLink" href="http://svn.apache.org/repos/asf/tika/trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml">tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml</a> in order to Tika can map the file extension with its MIME-Type. You should add something like this:</p>
+<div>
+<pre> <mime-type type="application/hello">
<glob pattern="*.hi"/>
- </mime-type></pre></div></div><div class="section"><h3><a name="Create_your_Parser_class">Create your Parser class</a></h3><p>Now, you need to create your new parser. This is a class that must implement the Parser interface offered by Tika. A very simple Tika Parser looks like this:</p><div><pre>/*
+ </mime-type></pre></div></div>
+<div class="section">
+<h3><a name="Create_your_Parser_class">Create your Parser class</a></h3>
+<p>Now, you need to create your new parser. This is a class that must implement the Parser interface offered by Tika. A very simple Tika Parser looks like this:</p>
+<div>
+<pre>/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
@@ -150,7 +172,13 @@ public class HelloParser implements Pars
throws IOException, SAXException, TikaException {
parse(stream, handler, metadata, new ParseContext());
}
-}</pre></div><p>Pay special attention to the definition of the SUPPORTED_TYPES static class field in the parser class that defines what MIME-Types it supports. </p><p>Is in the "parse" method where you will do all your work. This is, extract the information of the resource and then set the metadata.</p></div><div class="section"><h3><a name="List_the_new_parser">List the new parser</a></h3><p>Finally, you should explicitly tell the AutoDetectParser to include your new parser. This step is only needed if you want to use the AutoDetectParser functionality. If you figure out the correct parser in a different way, it isn't needed. </p><p>List your new parser in: <a class="externalLink" href="http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/main/resources/META-INF/services/org.apache.tika.parser.Parser">tika-parsers/src/main/resources/META-INF/services/org.apache.tika.parser.Parser</a></p></div></div>
+}</pre></div>
+<p>Pay special attention to the definition of the SUPPORTED_TYPES static class field in the parser class that defines what MIME-Types it supports. </p>
+<p>Is in the "parse" method where you will do all your work. This is, extract the information of the resource and then set the metadata.</p></div>
+<div class="section">
+<h3><a name="List_the_new_parser">List the new parser</a></h3>
+<p>Finally, you should explicitly tell the AutoDetectParser to include your new parser. This step is only needed if you want to use the AutoDetectParser functionality. If you figure out the correct parser in a different way, it isn't needed. </p>
+<p>List your new parser in: <a class="externalLink" href="http://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/main/resources/META-INF/services/org.apache.tika.parser.Parser">tika-parsers/src/main/resources/META-INF/services/org.apache.tika.parser.Parser</a></p></div></div>
</div>
<div id="sidebar">
<div id="navigation">