You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by ju...@apache.org on 2010/08/05 16:19:18 UTC
svn commit: r982632 [1/3] - in /tika/site: publish/ publish/0.5/
publish/0.6/ publish/0.7/ src/site/apt/ src/site/apt/0.5/ src/site/apt/0.6/
src/site/apt/0.7/
Author: jukka
Date: Thu Aug 5 14:19:17 2010
New Revision: 982632
URL: http://svn.apache.org/viewvc?rev=982632&view=rev
Log:
TIKA-473: Prepare Tika site for svnpubsub
Fix local links that became invalid with a more recent site plugin version
Modified:
tika/site/publish/0.5/documentation.html
tika/site/publish/0.5/formats.html
tika/site/publish/0.6/formats.html
tika/site/publish/0.6/parser.html
tika/site/publish/0.7/formats.html
tika/site/publish/0.7/parser.html
tika/site/publish/0.7/parser_guide.html
tika/site/publish/index.html
tika/site/src/site/apt/0.5/documentation.apt
tika/site/src/site/apt/0.5/formats.apt
tika/site/src/site/apt/0.6/formats.apt
tika/site/src/site/apt/0.6/parser.apt
tika/site/src/site/apt/0.7/formats.apt
tika/site/src/site/apt/0.7/parser.apt
tika/site/src/site/apt/0.7/parser_guide.apt
tika/site/src/site/apt/index.apt
Modified: tika/site/publish/0.5/documentation.html
URL: http://svn.apache.org/viewvc/tika/site/publish/0.5/documentation.html?rev=982632&r1=982631&r2=982632&view=diff
==============================================================================
--- tika/site/publish/0.5/documentation.html (original)
+++ tika/site/publish/0.5/documentation.html Thu Aug 5 14:19:17 2010
@@ -192,8 +192,8 @@
</div>
<div id="bodyColumn">
<div id="contentBox">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Apache Tika Documentation<a name="Apache_Tika_Documentation"></a></h2><p>This document descri
bes the key abstractions and usage of Apache Tika.</p></div><div class="section"><h2>The Parser interface<a name="The_Parser_interface"></a></h2><p>The <a href="#apidocsorgapachetikaparserParser.html">org.apache.tika.parser.Parser</a> interface is the key concept of Apache Tika. It hides the complexity of different file formats and parsing libraries while providing a simple and powerful mechanism for client applications to extract structured text content and metadata from all sorts of documents. All this is achieved with a single method:</p><div><pre>void parse(InputStream stream, ContentHandler handler, Metadata metadata)
- throws IOException, SAXException, TikaException;</pre></div><p>The <tt>parse</tt> method takes the document to be parsed and related metadata as input and outputs the results as XHTML SAX events and extra metadata. The main criteria that lead to this design were:</p><dl><dt>Streamed parsing</dt><dd>The interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. This allows even huge documents to be parsed without excessive resource requirements.</dd><dt>Structured content</dt><dd>A parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. A client application can use this information for example to better judge the relevance of different parts of the parsed document.</dd><dt>Input metadata</dt><dd>A client application should be able to include metadata like the file name or declared content type with the document to be p
arsed. The parser implementation can use this information to better guide the parsing process.</dd><dt>Output metadata</dt><dd>A parser implementation should be able to return document metadata in addition to document content. Many document formats contain metadata like the name of the author that may be useful to client applications.</dd></dl><p>These criteria are reflected in the arguments of the <tt>parse</tt> method.</p></div><div class="section"><h2>Document input stream<a name="Document_input_stream"></a></h2><p>The first argument is an <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html">InputStream</a> for reading the document to be parsed.</p><p>If this document stream can not be read, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html">IOException</a> is passed up to the client application. If the stream can be read but not parsed (for example if
the document is corrupted), then the parser throws a <a href="#apidocsorgapachetikaexceptionTikaException.html">TikaException</a>.</p><p>The parser implementation will consume this stream but <i>will not close it</i>. Closing the stream is the responsibility of the client application that opened it in the first place. The recommended pattern for using streams with the <tt>parse</tt> method is:</p><div><pre>InputStream stream = ...; // open the stream
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Apache Tika Documentation<a name="Apache_Tika_Documentation"></a></h2><p>This document descri
bes the key abstractions and usage of Apache Tika.</p></div><div class="section"><h2>The Parser interface<a name="The_Parser_interface"></a></h2><p>The <a href="./api/org/apache/tika/parser/Parser.html">org.apache.tika.parser.Parser</a> interface is the key concept of Apache Tika. It hides the complexity of different file formats and parsing libraries while providing a simple and powerful mechanism for client applications to extract structured text content and metadata from all sorts of documents. All this is achieved with a single method:</p><div><pre>void parse(InputStream stream, ContentHandler handler, Metadata metadata)
+ throws IOException, SAXException, TikaException;</pre></div><p>The <tt>parse</tt> method takes the document to be parsed and related metadata as input and outputs the results as XHTML SAX events and extra metadata. The main criteria that lead to this design were:</p><dl><dt>Streamed parsing</dt><dd>The interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. This allows even huge documents to be parsed without excessive resource requirements.</dd><dt>Structured content</dt><dd>A parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. A client application can use this information for example to better judge the relevance of different parts of the parsed document.</dd><dt>Input metadata</dt><dd>A client application should be able to include metadata like the file name or declared content type with the document to be p
arsed. The parser implementation can use this information to better guide the parsing process.</dd><dt>Output metadata</dt><dd>A parser implementation should be able to return document metadata in addition to document content. Many document formats contain metadata like the name of the author that may be useful to client applications.</dd></dl><p>These criteria are reflected in the arguments of the <tt>parse</tt> method.</p></div><div class="section"><h2>Document input stream<a name="Document_input_stream"></a></h2><p>The first argument is an <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html">InputStream</a> for reading the document to be parsed.</p><p>If this document stream can not be read, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html">IOException</a> is passed up to the client application. If the stream can be read but not parsed (for example if
the document is corrupted), then the parser throws a <a href="./api/org/apache/tika/exception/TikaException.html">TikaException</a>.</p><p>The parser implementation will consume this stream but <i>will not close it</i>. Closing the stream is the responsibility of the client application that opened it in the first place. The recommended pattern for using streams with the <tt>parse</tt> method is:</p><div><pre>InputStream stream = ...; // open the stream
try {
parser.parse(stream, ...); // parse the stream
} finally {
@@ -205,14 +205,14 @@ try {
<body>
...
</body>
-</html></pre></div><p>Parser implementations typically use the <a href="#apidocsorgapachetikasaxXHTMLContentHandler.html">XHTMLContentHandler</a> utility class to generate the XHTML output.</p><p>Dealing with the raw SAX events can be a bit complex, so Apache Tika (since version 0.2) comes with a number of utility classes that can be used to process and convert the event stream to other representations.</p><p>For example, the <a href="#apidocsorgapachetikasaxBodyContentHandler.html">BodyContentHandler</a> class can be used to extract just the body part of the XHTML output and feed it either as SAX events to another content handler or as characters to an output stream, a writer, or simply a string. The following code snippet parses a document from the standard input stream and outputs the extracted text content to standard output:</p><div><pre>ContentHandler handler = new BodyContentHandler(System.out);
-parser.parse(System.in, handler, ...);</pre></div><p>Another useful class is <a href="#apidocsorgapachetikaparserParsingReader.html">ParsingReader</a> that uses a background thread to parse the document and returns the extracted text content as a character stream:</p><div><pre>InputStream stream = ...; // the document to be parsed
+</html></pre></div><p>Parser implementations typically use the <a href="./api/org/apache/tika/sax/XHTMLContentHandler.html">XHTMLContentHandler</a> utility class to generate the XHTML output.</p><p>Dealing with the raw SAX events can be a bit complex, so Apache Tika (since version 0.2) comes with a number of utility classes that can be used to process and convert the event stream to other representations.</p><p>For example, the <a href="./api/org/apache/tika/sax/BodyContentHandler.html">BodyContentHandler</a> class can be used to extract just the body part of the XHTML output and feed it either as SAX events to another content handler or as characters to an output stream, a writer, or simply a string. The following code snippet parses a document from the standard input stream and outputs the extracted text content to standard output:</p><div><pre>ContentHandler handler = new BodyContentHandler(System.out);
+parser.parse(System.in, handler, ...);</pre></div><p>Another useful class is <a href="./api/org/apache/tika/parser/ParsingReader.html">ParsingReader</a> that uses a background thread to parse the document and returns the extracted text content as a character stream:</p><div><pre>InputStream stream = ...; // the document to be parsed
Reader reader = new ParsingReader(parser, stream, ...);
try {
...; // read the document text using the reader
} finally {
reader.close(); // the document stream is closed automatically
-}</pre></div></div><div class="section"><h2>Document metadata<a name="Document_metadata"></a></h2><p>The final argument to the <tt>parse</tt> method is used to pass document metadata both in and out of the parser. Document metadata is expressed as an <a href="#apidocsorgapachetikametadataMetadata.html">Metadata</a> object.</p><p>The following are some of the more interesting metadata properties:</p><dl><dt>Metadata.RESOURCE_NAME_KEY</dt><dd>The name of the file or resource that contains the document.<p>A client application can set this property to allow the parser to use file name heuristics to determine the format of the document.</p><p>The parser implementation may set this property if the file format contains the canonical name of the file (for example the Gzip format has a slot for the file name).</p></dd><dt>Metadata.CONTENT_TYPE</dt><dd>The declared content type of the document.<p>A client application can set this property based on for example a HTTP Content-Type heade
r. The declared content type may help the parser to correctly interpret the document.</p><p>The parser implementation sets this property to the content type according to which the document was parsed.</p></dd><dt>Metadata.TITLE</dt><dd>The title of the document.<p>The parser implementation sets this property if the document format contains an explicit title field.</p></dd><dt>Metadata.AUTHOR</dt><dd>The name of the author of the document.<p>The parser implementation sets this property if the document format contains an explicit author field.</p></dd></dl><p>Note that metadata handling is still being discussed by the Tika development team, and it is likely that there will be some (backwards incompatible) changes in metadata handling before Tika 1.0.</p></div><div class="section"><h2>Parser implementations<a name="Parser_implementations"></a></h2><p>Apache Tika comes with a number of parser classes for parsing <a href="#formats.html">various document formats</a>. You can also
extend Tika with your own parsers, and of course any contributions to Tika are warmly welcome.</p><p>The goal of Tika is to reuse existing parser libraries like <a class="externalLink" href="http://www.pdfbox.org/">PDFBox</a> or <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> as much as possible, and so most of the parser classes in Tika are adapters to such external libraries.</p><p>Tika also contains some general purpose parser implementations that are not targeted at any specific document formats. The most notable of these is the <a href="#apidocsorgapachetikaparserAutoDetectParser.html">AutoDetectParser</a> class that encapsulates all Tika functionality into a single parser that can handle any types of documents. This parser will automatically determine the type of the incoming document based on various heuristics and will then parse the document accordingly.</p></div>
+}</pre></div></div><div class="section"><h2>Document metadata<a name="Document_metadata"></a></h2><p>The final argument to the <tt>parse</tt> method is used to pass document metadata both in and out of the parser. Document metadata is expressed as an <a href="./api/org/apache/tika/metadata/Metadata.html">Metadata</a> object.</p><p>The following are some of the more interesting metadata properties:</p><dl><dt>Metadata.RESOURCE_NAME_KEY</dt><dd>The name of the file or resource that contains the document.<p>A client application can set this property to allow the parser to use file name heuristics to determine the format of the document.</p><p>The parser implementation may set this property if the file format contains the canonical name of the file (for example the Gzip format has a slot for the file name).</p></dd><dt>Metadata.CONTENT_TYPE</dt><dd>The declared content type of the document.<p>A client application can set this property based on for example a HTTP Content-Type hea
der. The declared content type may help the parser to correctly interpret the document.</p><p>The parser implementation sets this property to the content type according to which the document was parsed.</p></dd><dt>Metadata.TITLE</dt><dd>The title of the document.<p>The parser implementation sets this property if the document format contains an explicit title field.</p></dd><dt>Metadata.AUTHOR</dt><dd>The name of the author of the document.<p>The parser implementation sets this property if the document format contains an explicit author field.</p></dd></dl><p>Note that metadata handling is still being discussed by the Tika development team, and it is likely that there will be some (backwards incompatible) changes in metadata handling before Tika 1.0.</p></div><div class="section"><h2>Parser implementations<a name="Parser_implementations"></a></h2><p>Apache Tika comes with a number of parser classes for parsing <a href="./formats.html">various document formats</a>. You can al
so extend Tika with your own parsers, and of course any contributions to Tika are warmly welcome.</p><p>The goal of Tika is to reuse existing parser libraries like <a class="externalLink" href="http://www.pdfbox.org/">PDFBox</a> or <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> as much as possible, and so most of the parser classes in Tika are adapters to such external libraries.</p><p>Tika also contains some general purpose parser implementations that are not targeted at any specific document formats. The most notable of these is the <a href="./api/org/apache/tika/parser/AutoDetectParser.html">AutoDetectParser</a> class that encapsulates all Tika functionality into a single parser that can handle any types of documents. This parser will automatically determine the type of the incoming document based on various heuristics and will then parse the document accordingly.</p></div>
</div>
</div>
<div class="clear">
Modified: tika/site/publish/0.5/formats.html
URL: http://svn.apache.org/viewvc/tika/site/publish/0.5/formats.html?rev=982632&r1=982631&r2=982632&view=diff
==============================================================================
--- tika/site/publish/0.5/formats.html (original)
+++ tika/site/publish/0.5/formats.html Thu Aug 5 14:19:17 2010
@@ -192,7 +192,7 @@
</div>
<div id="bodyColumn">
<div id="contentBox">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Supported Document Formats<a name="Supported_Document_Formats"></a></h2><p>This page lists al
l the document formats supported by Apache Tika.</p><div class="section"><h3>Microsoft's OLE 2 Compound Document format<a name="Microsofts_OLE_2_Compound_Document_format"></a></h3><p>A number of Microsoft applications, most notably the Microsoft Office suite, use the generic OLE 2 Compound Document format as the basis of their document formats. Tika uses <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> to support a number of these formats.</p><p>The OLE2 Compound Document format is designed for use with random access files, and so the input stream passed to a Tika parser needs to be spooled in memory or in a temporary file depending on the size of the document. See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-153">TIKA-153</a> for an effort to avoid this extra temporary file if the input document already comes from a file.</p><p>In addition to the shared base format there's also a shared sets of metadata in typical OLE2 documen
ts. Tika uses the <a class="externalLink" href="http://poi.apache.org/hpsf/">HPSF library</a> from POI to parse these property sets and exposes them as the following document metadata:</p><ul><li><tt>TITLE</tt> Title</li><li><tt>SUBJECT</tt> Subject</li><li><tt>AUTHOR</tt> Author</li><li><tt>KEYWORDS</tt> Keywords</li><li><tt>COMMENTS</tt> Comments</li><li><tt>TEMPLATE</tt> Template</li><li><tt>LAST_SAVED</tt> Last Saved By</li><li><tt>REVISION_NUMBER</tt> Revision Number</li><li><tt>LAST_PRINTED</tt> Last Printed</li><li><tt>LAST_SAVED</tt> Last Saved Time/Date</li><li><tt>LAST_SAVED</tt> Last Saved Time/Date</li><li><tt>PAGE_COUNT</tt> Number of Pages</li><li><tt>WORD_COUNT</tt> Number of Words</li><li><tt>CHARACTER_COUNT</tt> Number of Characters</li><li><tt>APPLICATION_NAME</tt> Name of Creating Application</li></ul><p>Note that in practice the metadata in many documents is either missing, incomplete or even incorrect, so a client application should not rely too much on
this information.</p><p>Support for the new Office Open XML format used by Microsoft Office version 2007 is pending for a POI upgrade. Current status is recorded in <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-152">TIKA-152</a>.</p><p>The generic OLE2 Compound Document format is automatically detected using a magic number, and further parsing can automatically determine the more specific document format. Tika also knows a number of common glob patterns like <tt>*.doc</tt> and <tt>*.ppt</tt> for these formats.</p><p>The supported OLE 2 Compound Document formats are:</p><dl><dt>Microsoft Excel (application/vnd.ms-excel)</dt><dd> Excel spreadsheet support is available in all versions of Tika and is based on the <a class="externalLink" href="http://poi.apache.org/hssf/">HSSF library</a> from POI.<p>The Excel parser in Tika uses the <a class="externalLink" href="http://poi.apache.org/hssf/how-to.html#event_api">HSSF event API</a> and is able to extract
much of the document structure, including all (non-empty) worksheets and their table structures. Formula results are extracted as stored in the Excel file, and cell links are exposed as XHTML links. These features were added in Tika version 0.2.</p><p>Cell comments and formatting are currently not supported. See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-148">TIKA-148</a> and <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-103">TIKA-103</a> for the respective issues.</p><p>See the <a href="#xref-testorgapachetikaparsermicrosoftExcelParserTest.html">ExcelParserTest</a> test case for an example of parsing Microsoft Excel files.</p></dd><dt>Microsoft Word (application/msword)</dt><dd> Word document support is available in all versions of Tika and is based on the <a class="externalLink" href="http://poi.apache.org/hwpf/">HWPF library</a> from POI.<p>The Word parser uses the <a class="externalLink" href="http://poi.apache.or
g/apidocs/org/apache/poi/hwpf/extractor/WordExtractor.html">WordExtractor</a> class from HWPF to extract document content as a sequence of paragraphs.</p><p>See the <a href="#xref-testorgapachetikaparsermicrosoftWordParserTest.html">WordParserTest</a> test case for an example of parsing Microsoft Word files.</p></dd><dt>Microsoft PowerPoint (application/vnd.ms-powerpoint)</dt><dd> PowerPoint presentation support is available in all versions of Tika and is based on the <a class="externalLink" href="http://poi.apache.org/hslf/">HSLF library</a> from POI.<p>The PowerPoint parser uses the <a class="externalLink" href="http://poi.apache.org/apidocs/org/apache/poi/hslf/extractor/PowerPointExtractor.html">PowerPointExtractor</a> class from HSLF to extract spreadsheet content as a single paragraph.</p><p>See the <a href="#xref-testorgapachetikaparsermicrosoftPowerPointParserTest.html">PowerPointParserTest</a> test case for an example of parsing Microsoft PowerPoint files.</p></dd><d
t>Microsoft Visio (application/vnd.visio)</dt><dd> Visio diagram support was added in Tika version 0.2 and is based on the <a class="externalLink" href="http://poi.apache.org/hdgf/">HDGF library</a> from POI.<p>The Visio parser uses the <a class="externalLink" href="http://poi.apache.org/apidocs/org/apache/poi/hdgf/extractor/VisioTextExtractor.html">VisioExtractor</a> class from HDGF to extract diagram content as a sequence of paragraphs.</p></dd><dt>Microsoft Outlook (application/vnd.ms-outlook)</dt><dd> Outlook message support was added in Tika version 0.2 and is based on the <a class="externalLink" href="http://poi.apache.org/hsmf/">HSMF library</a> from POI.<p>The Outlook parser extracts the subject of the message and the From, To, Cc, and Bcc addresses (formatted for display) along with the body text of text/plain messages. The <tt>AUTHOR</tt>, <tt>TITLE</tt> and <tt>SUBJECT</tt> metadata properties are set explicitly, overriding potential generic document metadata retr
ieved from OLE2 property sets.</p></dd></dl></div><div class="section"><h3>Compression formats<a name="Compression_formats"></a></h3><p>General purpose compression formats are used to reduce the size of any kinds of documents. Tika uses a parsing pipeline to support general purpose compression: in the first stage the compressed stream decompressed and the resulting decompressed stream is passed on to a second parsing stage where it will be processed as if the document had never been compressed.</p><p>Tika contains magic numbers and glob patterns for auto-detecting all supported compression formats. The glob patterns of compression formats are also used to determine the name of the original uncompressed document. If a client application has supplied a <tt>RESOURCE_NAME_KEY</tt> metadata property that matches such a glob pattern, then the decompressing first parsing stage will replace the <tt>RESOURCE_NAME_KEY</tt> metadata property with the deduced original document name befo
re passing control to the second parsing stage.</p><p>Note that apart from the special handling of the <tt>RESOURCE_NAME_KEY</tt> property, no document metadata is passed to or from the second parsing stage. Only the text content extracted by the second stage parser is returned to the client application.</p><p>The supported compression formats are:</p><dl><dt>gzip compression (application/x-gzip)</dt><dd> <a class="externalLink" href="http://en.wikipedia.org/wiki/Gzip">Gzip</a> support was added in Tika version 0.2 and is based on the <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/util/zip/GZIPInputStream.html">GZIPInputStream</a> class in the Java 5 class library.<p>The known gzip glob patterns are <tt>*.tgz</tt>, <tt>*.gz</tt> and <tt>*-gz</tt>, and they will respectively be replaced with <tt>*.tar</tt>, <tt>*</tt> and <tt>*</tt> as described above.</p></dd><dt>bzip2 compression (application/x-bzip)</dt><dd> <a class="externalLink" href="http://
en.wikipedia.org/wiki/Bzip2">Bzip2</a> support was added in Tika version 0.2 and is based on bzip2 parsing code from <a class="externalLink" href="http://ant.apache.org/">Apache Ant</a>, which in turn was originally based on work by Keiron Liddle from Aftex Software.<p>The known bzip2 glob patterns are <tt>*.tbz</tt>, <tt>*.tbz2</tt>, <tt>*.bz</tt> and <tt>*.bz2</tt>, and they will respectively be replaced with <tt>*.tar</tt>, <tt>*.tar</tt>, <tt>*</tt> and <tt>*</tt> as described above.</p></dd></dl></div><div class="section"><h3>Audio formats<a name="Audio_formats"></a></h3><p>Tika can detect several common audio formats and extract metadata from them. Text extraction is supported for some MIDI-based karaoke formats that contain the lyrics of the encoded audio.</p><p>See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-94">TIKA-94</a> for an effort to integrate speech recognition support to Tika.</p><dl><dt>MP3 Audio (audio/mpeg)</dt><dd> The parsin
g of <a class="externalLink" href="http://www.id3.org/ID3v1">ID3v1</a> tags from MP3 files was added in Tika version 0.2. If found the following metadata is extracted and set:<ul><li><tt>TITLE</tt> Title</li><li><tt>SUBJECT</tt> Subject</li></ul><p>The above information, as well as the <tt>Album</tt>, <tt>Track</tt>, <tt>Year</tt>, <tt>Genre</tt> and additional <tt>Comment</tt> are extracted when set in the file.</p></dd><dt>MIDI audio (audio/midi)</dt><dd> Tika uses the MIDI support in <tt>javax.audio.midi</tt> to parse MIDI sequence files. Many karaoke file formats are based on MIDI, and contain lyrics as embedded text tracks that Tika knows how to extract.<p>Support for MIDI files was added in Tika 0.3.</p></dd><dt>Wave audio (audio/basic)</dt><dd> Tika supports sampled wave audio (.wav files, etc.) using the <tt>javax.audio.sampled</tt> package. Only sampling metadata is extracted.<p>Support for sampled wave audio was added in Tika 0.3. </p></dd></dl></div><div class="se
ction"><h3>Other supported formats<a name="Other_supported_formats"></a></h3><dl><dt>Extensible Markup Language (application/xml)</dt><dd> Tika uses the <tt>javax.xml</tt> classes to parse Extensible Markup Language files. Support for Extensible Markup Language files was added in Tika 0.1.</dd><dt>HyperText Markup Language (text/html)</dt><dd> Tika uses the <a class="externalLink" href="http://sourceforge.net/projects/nekohtml">CyberNeko</a> library to parse HyperText Markup Language files. Support for HyperText Markup Language files was added in Tika 0.1.</dd><dt>Images (image/*)</dt><dd> Tika uses the <tt>javax.imageio</tt> classes to extract metadata from image files.<p>Support for Image files was added in Tika 0.2.</p></dd><dt>Java class files</dt><dd> The parsing of Java Class files is based on the asm library and work by Dave Brosius in JCR-1522.<p>Support for Java Class files was added in Tika 0.2.</p></dd><dt>Java jar archives</dt><dd> The parsing of Java JAR archive
s is performed using a combination of the ZIP and Java class file parsers.<p>Support for Java JAR archives was added in Tika 0.2.</p></dd><dt>OpenDocument (application/vnd.oasis.opendocument.*)</dt><dd> Tika uses the built-in ZIP and XML features in Java to parse the <a class="externalLink" href="http://en.wikipedia.org/wiki/OpenDocument">OpenDocument</a> document types used most notably by OpenOffice 2.0 and higher. The older OpenOffice 1.0 formats are also supported, though they are currently not auto-detected as well as the newer formats.<p>Support for the OpenDocument formats was added in Tika 0.3.</p></dd><dt>Plain text (text/plain)</dt><dd> Tika uses the <a class="externalLink" href="http://www.icu-project.org/">International Components for Unicode</a> Java library (ICU4J) to parse plain text. Support for plain text was added in Tika 0.1.<p>Extracting text content from plain text files is actually a relatively complex task due to the fact that the character encoding of
the text file is often unknown to the parser.</p><p>The text parser in Tika uses the ICU4J <a class="externalLink" href="http://www.icu-project.org/apiref/icu4j/com/ibm/icu/text/CharsetDetector.html">CharsetDetector</a> class to automatically detect the character encoding of any text input. As an added benefit, the ICU4J library is in some cases able to detect also the language in which the text is written.</p><p>The character encoding and language of the plain text document are returned as the <tt>Metadata.CONTENT_ENCODING</tt> and <tt>Metadata.LANGUAGE</tt> metadata properties. If the (declared) content encoding of a text document is already known to the client application, then it can be supplied as the <tt>Metadata.CONTENT_ENCODING</tt> metadata property to the parser to simplify encoding detection.</p></dd><dt>Portable Document Format (application/pdf)</dt><dd> Tika uses the <a class="externalLink" href="http://www.pdfbox.org">PDFBox</a> library to parse Portable Docum
ent Format (PDF) documents.<p>Support for PDF was added in Tika 0.1.</p></dd><dt>Rich Text Format (application/rtf)</dt><dd> Tika uses Java's built-in Swing library to parse Rich Text Format (RTF) documents. Support for RTF was added in Tika 0.1.<p>The RTF parser in Tika uses the Swing <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/javax/swing/text/rtf/RTFEditorKit.html">RTFEditorKit</a> class to extract all text from an RTF document as a single paragraph. Document metadata extraction is currently not supported.</p></dd><dt>tar archive (application/x-tar)</dt><dd> Tika uses an adapted version of the tar parsing code from <a class="externalLink" href="http://ant.apache.org/">Apache Ant</a> to parse tar archives. The tar code is originally based on work by Timothy Gerard Endres.<p>Support for tar archives was added in Tika 0.2.</p></dd><dt>ZIP archive (application/zip)</dt><dd> Tika uses Java's built-in Zip classes to parse ZIP files.<p>Support for ZIP w
as added in Tika 0.2.</p></dd></dl></div></div>
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Supported Document Formats<a name="Supported_Document_Formats"></a></h2><p>This page lists al
l the document formats supported by Apache Tika.</p><div class="section"><h3>Microsoft's OLE 2 Compound Document format<a name="Microsofts_OLE_2_Compound_Document_format"></a></h3><p>A number of Microsoft applications, most notably the Microsoft Office suite, use the generic OLE 2 Compound Document format as the basis of their document formats. Tika uses <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> to support a number of these formats.</p><p>The OLE2 Compound Document format is designed for use with random access files, and so the input stream passed to a Tika parser needs to be spooled in memory or in a temporary file depending on the size of the document. See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-153">TIKA-153</a> for an effort to avoid this extra temporary file if the input document already comes from a file.</p><p>In addition to the shared base format there's also a shared sets of metadata in typical OLE2 documen
ts. Tika uses the <a class="externalLink" href="http://poi.apache.org/hpsf/">HPSF library</a> from POI to parse these property sets and exposes them as the following document metadata:</p><ul><li><tt>TITLE</tt> Title</li><li><tt>SUBJECT</tt> Subject</li><li><tt>AUTHOR</tt> Author</li><li><tt>KEYWORDS</tt> Keywords</li><li><tt>COMMENTS</tt> Comments</li><li><tt>TEMPLATE</tt> Template</li><li><tt>LAST_SAVED</tt> Last Saved By</li><li><tt>REVISION_NUMBER</tt> Revision Number</li><li><tt>LAST_PRINTED</tt> Last Printed</li><li><tt>LAST_SAVED</tt> Last Saved Time/Date</li><li><tt>LAST_SAVED</tt> Last Saved Time/Date</li><li><tt>PAGE_COUNT</tt> Number of Pages</li><li><tt>WORD_COUNT</tt> Number of Words</li><li><tt>CHARACTER_COUNT</tt> Number of Characters</li><li><tt>APPLICATION_NAME</tt> Name of Creating Application</li></ul><p>Note that in practice the metadata in many documents is either missing, incomplete or even incorrect, so a client application should not rely too much on
this information.</p><p>Support for the new Office Open XML format used by Microsoft Office version 2007 is pending for a POI upgrade. Current status is recorded in <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-152">TIKA-152</a>.</p><p>The generic OLE2 Compound Document format is automatically detected using a magic number, and further parsing can automatically determine the more specific document format. Tika also knows a number of common glob patterns like <tt>*.doc</tt> and <tt>*.ppt</tt> for these formats.</p><p>The supported OLE 2 Compound Document formats are:</p><dl><dt>Microsoft Excel (application/vnd.ms-excel)</dt><dd> Excel spreadsheet support is available in all versions of Tika and is based on the <a class="externalLink" href="http://poi.apache.org/hssf/">HSSF library</a> from POI.<p>The Excel parser in Tika uses the <a class="externalLink" href="http://poi.apache.org/hssf/how-to.html#event_api">HSSF event API</a> and is able to extract
much of the document structure, including all (non-empty) worksheets and their table structures. Formula results are extracted as stored in the Excel file, and cell links are exposed as XHTML links. These features were added in Tika version 0.2.</p><p>Cell comments and formatting are currently not supported. See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-148">TIKA-148</a> and <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-103">TIKA-103</a> for the respective issues.</p></dd><dt>Microsoft Word (application/msword)</dt><dd> Word document support is available in all versions of Tika and is based on the <a class="externalLink" href="http://poi.apache.org/hwpf/">HWPF library</a> from POI.<p>The Word parser uses the <a class="externalLink" href="http://poi.apache.org/apidocs/org/apache/poi/hwpf/extractor/WordExtractor.html">WordExtractor</a> class from HWPF to extract document content as a sequence of paragraphs.</p></dd><dt
>Microsoft PowerPoint (application/vnd.ms-powerpoint)</dt><dd> PowerPoint presentation support is available in all versions of Tika and is based on the <a class="externalLink" href="http://poi.apache.org/hslf/">HSLF library</a> from POI.<p>The PowerPoint parser uses the <a class="externalLink" href="http://poi.apache.org/apidocs/org/apache/poi/hslf/extractor/PowerPointExtractor.html">PowerPointExtractor</a> class from HSLF to extract spreadsheet content as a single paragraph.</p></dd><dt>Microsoft Visio (application/vnd.visio)</dt><dd> Visio diagram support was added in Tika version 0.2 and is based on the <a class="externalLink" href="http://poi.apache.org/hdgf/">HDGF library</a> from POI.<p>The Visio parser uses the <a class="externalLink" href="http://poi.apache.org/apidocs/org/apache/poi/hdgf/extractor/VisioTextExtractor.html">VisioExtractor</a> class from HDGF to extract diagram content as a sequence of paragraphs.</p></dd><dt>Microsoft Outlook (application/vnd.ms-outlo
ok)</dt><dd> Outlook message support was added in Tika version 0.2 and is based on the <a class="externalLink" href="http://poi.apache.org/hsmf/">HSMF library</a> from POI.<p>The Outlook parser extracts the subject of the message and the From, To, Cc, and Bcc addresses (formatted for display) along with the body text of text/plain messages. The <tt>AUTHOR</tt>, <tt>TITLE</tt> and <tt>SUBJECT</tt> metadata properties are set explicitly, overriding potential generic document metadata retrieved from OLE2 property sets.</p></dd></dl></div><div class="section"><h3>Compression formats<a name="Compression_formats"></a></h3><p>General purpose compression formats are used to reduce the size of any kinds of documents. Tika uses a parsing pipeline to support general purpose compression: in the first stage the compressed stream decompressed and the resulting decompressed stream is passed on to a second parsing stage where it will be processed as if the document had never been compressed
.</p><p>Tika contains magic numbers and glob patterns for auto-detecting all supported compression formats. The glob patterns of compression formats are also used to determine the name of the original uncompressed document. If a client application has supplied a <tt>RESOURCE_NAME_KEY</tt> metadata property that matches such a glob pattern, then the decompressing first parsing stage will replace the <tt>RESOURCE_NAME_KEY</tt> metadata property with the deduced original document name before passing control to the second parsing stage.</p><p>Note that apart from the special handling of the <tt>RESOURCE_NAME_KEY</tt> property, no document metadata is passed to or from the second parsing stage. Only the text content extracted by the second stage parser is returned to the client application.</p><p>The supported compression formats are:</p><dl><dt>gzip compression (application/x-gzip)</dt><dd> <a class="externalLink" href="http://en.wikipedia.org/wiki/Gzip">Gzip</a> support was add
ed in Tika version 0.2 and is based on the <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/util/zip/GZIPInputStream.html">GZIPInputStream</a> class in the Java 5 class library.<p>The known gzip glob patterns are <tt>*.tgz</tt>, <tt>*.gz</tt> and <tt>*-gz</tt>, and they will respectively be replaced with <tt>*.tar</tt>, <tt>*</tt> and <tt>*</tt> as described above.</p></dd><dt>bzip2 compression (application/x-bzip)</dt><dd> <a class="externalLink" href="http://en.wikipedia.org/wiki/Bzip2">Bzip2</a> support was added in Tika version 0.2 and is based on bzip2 parsing code from <a class="externalLink" href="http://ant.apache.org/">Apache Ant</a>, which in turn was originally based on work by Keiron Liddle from Aftex Software.<p>The known bzip2 glob patterns are <tt>*.tbz</tt>, <tt>*.tbz2</tt>, <tt>*.bz</tt> and <tt>*.bz2</tt>, and they will respectively be replaced with <tt>*.tar</tt>, <tt>*.tar</tt>, <tt>*</tt> and <tt>*</tt> as described above.</p></
dd></dl></div><div class="section"><h3>Audio formats<a name="Audio_formats"></a></h3><p>Tika can detect several common audio formats and extract metadata from them. Text extraction is supported for some MIDI-based karaoke formats that contain the lyrics of the encoded audio.</p><p>See <a class="externalLink" href="https://issues.apache.org/jira/browse/TIKA-94">TIKA-94</a> for an effort to integrate speech recognition support to Tika.</p><dl><dt>MP3 Audio (audio/mpeg)</dt><dd> The parsing of <a class="externalLink" href="http://www.id3.org/ID3v1">ID3v1</a> tags from MP3 files was added in Tika version 0.2. If found the following metadata is extracted and set:<ul><li><tt>TITLE</tt> Title</li><li><tt>SUBJECT</tt> Subject</li></ul><p>The above information, as well as the <tt>Album</tt>, <tt>Track</tt>, <tt>Year</tt>, <tt>Genre</tt> and additional <tt>Comment</tt> are extracted when set in the file.</p></dd><dt>MIDI audio (audio/midi)</dt><dd> Tika uses the MIDI support in <tt>ja
vax.audio.midi</tt> to parse MIDI sequence files. Many karaoke file formats are based on MIDI, and contain lyrics as embedded text tracks that Tika knows how to extract.<p>Support for MIDI files was added in Tika 0.3.</p></dd><dt>Wave audio (audio/basic)</dt><dd> Tika supports sampled wave audio (.wav files, etc.) using the <tt>javax.audio.sampled</tt> package. Only sampling metadata is extracted.<p>Support for sampled wave audio was added in Tika 0.3. </p></dd></dl></div><div class="section"><h3>Other supported formats<a name="Other_supported_formats"></a></h3><dl><dt>Extensible Markup Language (application/xml)</dt><dd> Tika uses the <tt>javax.xml</tt> classes to parse Extensible Markup Language files. Support for Extensible Markup Language files was added in Tika 0.1.</dd><dt>HyperText Markup Language (text/html)</dt><dd> Tika uses the <a class="externalLink" href="http://sourceforge.net/projects/nekohtml">CyberNeko</a> library to parse HyperText Markup Language files. Su
pport for HyperText Markup Language files was added in Tika 0.1.</dd><dt>Images (image/*)</dt><dd> Tika uses the <tt>javax.imageio</tt> classes to extract metadata from image files.<p>Support for Image files was added in Tika 0.2.</p></dd><dt>Java class files</dt><dd> The parsing of Java Class files is based on the asm library and work by Dave Brosius in JCR-1522.<p>Support for Java Class files was added in Tika 0.2.</p></dd><dt>Java jar archives</dt><dd> The parsing of Java JAR archives is performed using a combination of the ZIP and Java class file parsers.<p>Support for Java JAR archives was added in Tika 0.2.</p></dd><dt>OpenDocument (application/vnd.oasis.opendocument.*)</dt><dd> Tika uses the built-in ZIP and XML features in Java to parse the <a class="externalLink" href="http://en.wikipedia.org/wiki/OpenDocument">OpenDocument</a> document types used most notably by OpenOffice 2.0 and higher. The older OpenOffice 1.0 formats are also supported, though they are currentl
y not auto-detected as well as the newer formats.<p>Support for the OpenDocument formats was added in Tika 0.3.</p></dd><dt>Plain text (text/plain)</dt><dd> Tika uses the <a class="externalLink" href="http://www.icu-project.org/">International Components for Unicode</a> Java library (ICU4J) to parse plain text. Support for plain text was added in Tika 0.1.<p>Extracting text content from plain text files is actually a relatively complex task due to the fact that the character encoding of the text file is often unknown to the parser.</p><p>The text parser in Tika uses the ICU4J <a class="externalLink" href="http://www.icu-project.org/apiref/icu4j/com/ibm/icu/text/CharsetDetector.html">CharsetDetector</a> class to automatically detect the character encoding of any text input. As an added benefit, the ICU4J library is in some cases able to detect also the language in which the text is written.</p><p>The character encoding and language of the plain text document are returned as t
he <tt>Metadata.CONTENT_ENCODING</tt> and <tt>Metadata.LANGUAGE</tt> metadata properties. If the (declared) content encoding of a text document is already known to the client application, then it can be supplied as the <tt>Metadata.CONTENT_ENCODING</tt> metadata property to the parser to simplify encoding detection.</p></dd><dt>Portable Document Format (application/pdf)</dt><dd> Tika uses the <a class="externalLink" href="http://www.pdfbox.org">PDFBox</a> library to parse Portable Document Format (PDF) documents.<p>Support for PDF was added in Tika 0.1.</p></dd><dt>Rich Text Format (application/rtf)</dt><dd> Tika uses Java's built-in Swing library to parse Rich Text Format (RTF) documents. Support for RTF was added in Tika 0.1.<p>The RTF parser in Tika uses the Swing <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/javax/swing/text/rtf/RTFEditorKit.html">RTFEditorKit</a> class to extract all text from an RTF document as a single paragraph. Document metad
ata extraction is currently not supported.</p></dd><dt>tar archive (application/x-tar)</dt><dd> Tika uses an adapted version of the tar parsing code from <a class="externalLink" href="http://ant.apache.org/">Apache Ant</a> to parse tar archives. The tar code is originally based on work by Timothy Gerard Endres.<p>Support for tar archives was added in Tika 0.2.</p></dd><dt>ZIP archive (application/zip)</dt><dd> Tika uses Java's built-in Zip classes to parse ZIP files.<p>Support for ZIP was added in Tika 0.2.</p></dd></dl></div></div>
</div>
</div>
<div class="clear">
Modified: tika/site/publish/0.6/formats.html
URL: http://svn.apache.org/viewvc/tika/site/publish/0.6/formats.html?rev=982632&r1=982631&r2=982632&view=diff
==============================================================================
--- tika/site/publish/0.6/formats.html (original)
+++ tika/site/publish/0.6/formats.html Thu Aug 5 14:19:17 2010
@@ -192,7 +192,7 @@
</div>
<div id="bodyColumn">
<div id="contentBox">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Supported Document Formats<a name="Supported_Document_Formats"></a></h2><p>This page lists al
l the document formats supported by Apache Tika 0.6. Follow the links to the various parser class javadocs for more detailed information about each document format and how it is parsed by Tika.</p><ul><li><a href="#Supported_Document_Formats">Supported Document Formats</a><ul><li><a href="#HyperText_Markup_Language">HyperText Markup Language</a></li><li><a href="#XML_and_derived_formats">XML and derived formats</a></li><li><a href="#Microsoft_Office_document_formats">Microsoft Office document formats</a></li><li><a href="#OpenDocument_Format">OpenDocument Format</a></li><li><a href="#Portable_Document_Format">Portable Document Format</a></li><li><a href="#Electronic_Publication_Format">Electronic Publication Format</a></li><li><a href="#Rich_Text_Format">Rich Text Format</a></li><li><a href="#Compression_and_packaging_formats">Compression and packaging formats</a></li><li><a href="#Text_formats">Text formats</a></li><li><a href="#Audio_formats">Audio formats</a></li><li><a h
ref="#Image_formats">Image formats</a></li><li><a href="#Video_formats">Video formats</a></li><li><a href="#Java_class_files_and_archives">Java class files and archives</a></li><li><a href="#The_mbox_format">The mbox format</a></li></ul></li></ul><div class="section"><h3><a name="HyperText_Markup_Language">HyperText Markup Language</a><a name="HyperText_Markup_Language"></a></h3><p>The HyperText Markup Language (HTML) is the lingua franca of the web. Tika uses the <a class="externalLink" href="http://home.ccil.org/~cowan/XML/tagsoup/">TagSoup</a> library to support virtually any kind of HTML found on the web. The output from the <a href="#apiorgapachetikaparserhtmlHtmlParser.html">HtmlParser</a> class is guaranteed to be well-formed and valid XHTML, and various heuristics are used to prevent things like inline scripts from cluttering the extracted text content.</p></div><div class="section"><h3><a name="XML_and_derived_formats">XML and derived formats</a><a name="XML_and_der
ived_formats"></a></h3><p>The Extensible Markup Language (XML) format is a generic format that can be used for all kinds of content. Tika has custom parsers for some widely used XML vocabularies like XHTML, OOXML and ODF, but the default <a href="#apiorgapachetikaparserxmlDcXMLParser.html">DcXMLParser</a> class simply extracts the text content of the document and ignores any XML structure. The only exception to this rule are Dublin Core metadata elements that are used for the document metadata.</p></div><div class="section"><h3><a name="Microsoft_Office_document_formats">Microsoft Office document formats</a><a name="Microsoft_Office_document_formats"></a></h3><p>Microsoft Office and some related applications produce documents in the generic OLE 2 Compound Document and Office Open XML (OOXML) formats. The older OLE 2 format was introduced in Microsoft Office version 97 and was the default format until Office version 2007 and the new XML-based OOXML format. The <a href="#apior
gapachetikaparsermicrosoftOfficeParser.html">OfficeParser</a> and <a href="#apiorgapachetikaparsermicrosoftooxmlOOXMLParser.html">OOXMLParser</a> classes use <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> libraries to support text and metadata extraction from both OLE2 and OOXML documents.</p></div><div class="section"><h3><a name="OpenDocument_Format">OpenDocument Format</a><a name="OpenDocument_Format"></a></h3><p>The OpenDocument format (ODF) is used most notably as the default format of the OpenOffice.org office suite. The <a href="#apiorgapachetikaparserodfOpenDocumentParser.html">OpenDocumentParser</a> class supports this format and the earlier OpenOffice 1.0 format on which ODF is based.</p></div><div class="section"><h3><a name="Portable_Document_Format">Portable Document Format</a><a name="Portable_Document_Format"></a></h3><p>The <a href="#apiorgapachetikaparserpdfPDFParser.html">PDFParser</a> class parsers Portable Document Format (PDF) docum
ents using the <a class="externalLink" href="http://pdfbox.apache.org/">Apache PDFBox</a> library.</p></div><div class="section"><h3><a name="Electronic_Publication_Format">Electronic Publication Format</a><a name="Electronic_Publication_Format"></a></h3><p>The <a href="#apiorgapachetikaparserepubEpubParser.html">EpubParser</a> class supports the Electronic Publication Format (EPUB) used for many digital books.</p></div><div class="section"><h3><a name="Rich_Text_Format">Rich Text Format</a><a name="Rich_Text_Format"></a></h3><p>The <a href="#apiorgapachetikaparserrtfRTFParser.html">RTFParser</a> class uses the standard javax.swing.text.rtf feature to extract text content from Rich Text Format (RTF) documents.</p></div><div class="section"><h3><a name="Compression_and_packaging_formats">Compression and packaging formats</a><a name="Compression_and_packaging_formats"></a></h3><p>Tika uses the <a class="externalLink" href="http://commons.apache.org/compress/">Commons Compress<
/a> library to support various compression and packaging formats. The <a href="#apiorgapachetikaparserpkgPackageParser.html">PackageParser</a> class and its subclasses first parse the top level compression or packaging format and then pass the unpacked document streams to a second parsing stage using the parser instance specified in the parse context.</p></div><div class="section"><h3><a name="Text_formats">Text formats</a><a name="Text_formats"></a></h3><p>Extracting text content from plain text files seems like a simple task until you start thinking of all the possible character encodings. The <a href="#apiorgapachetikaparsertxtTXTParser.html">TXTParser</a> class uses encoding detection code from the <a class="externalLink" href="http://site.icu-project.org/">ICU</a> project to automatically detect the character encoding of a text document.</p></div><div class="section"><h3><a name="Audio_formats">Audio formats</a><a name="Audio_formats"></a></h3><p>Tika can detect several
common audio formats and extract metadata from them. Even text extraction is supported for some audio files that contain lyrics or other textual content. The <a href="#apiorgapachetikaparseraudioAudioParser.html">AudioParser</a> and <a href="#apiorgapachetikaparseraudioMidiParser.html">MidiParser</a> classes use standard javax.sound features to process simple audio formats, and the <a href="#apiorgapachetikaparsermp3Mp3Parser.html">Mp3Parser</a> class adds support for the widely used MP3 format.</p></div><div class="section"><h3><a name="Image_formats">Image formats</a><a name="Image_formats"></a></h3><p>The <a href="#apiorgapachetikaparserimageImageParser.html">ImageParser</a> class uses the standard javax.imageio feature to extract simple metadata from image formats supported by the Java platform. More complex image metadata is available through the <a href="#apiorgapachetikaparserjpegJpegParser.html">JpegParser</a> class that uses the metadata-extractor library to suppor
ts Exif metadata extraction from Jpeg images.</p></div><div class="section"><h3><a name="Video_formats">Video formats</a><a name="Video_formats"></a></h3><p>Currently Tika only supports the Flash video format using a simple parsing algorithm implemented in the <a href="#apiorgapachetikaparserflvFLVParser">FLVParser</a> class.</p></div><div class="section"><h3><a name="Java_class_files_and_archives">Java class files and archives</a><a name="Java_class_files_and_archives"></a></h3><p>The <a href="#apiorgapachetikaparserasmClassParser">ClassParser</a> class extracts class names and method signatures from Java class files, and the <a href="#apiorgapachetikaparserpkgZipParser.html">ZipParser</a> class supports also jar archives.</p></div><div class="section"><h3><a name="The_mbox_format">The mbox format</a><a name="The_mbox_format"></a></h3><p>The <a href="#apiorgapachetikaparsermboxMboxParser.html">MboxParser</a> can extract email messages from the mbox format used by many email
archives and Unix-style mailboxes.</p></div></div>
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>Supported Document Formats<a name="Supported_Document_Formats"></a></h2><p>This page lists al
l the document formats supported by Apache Tika 0.6. Follow the links to the various parser class javadocs for more detailed information about each document format and how it is parsed by Tika.</p><ul><li><a href="#Supported_Document_Formats">Supported Document Formats</a><ul><li><a href="#HyperText_Markup_Language">HyperText Markup Language</a></li><li><a href="#XML_and_derived_formats">XML and derived formats</a></li><li><a href="#Microsoft_Office_document_formats">Microsoft Office document formats</a></li><li><a href="#OpenDocument_Format">OpenDocument Format</a></li><li><a href="#Portable_Document_Format">Portable Document Format</a></li><li><a href="#Electronic_Publication_Format">Electronic Publication Format</a></li><li><a href="#Rich_Text_Format">Rich Text Format</a></li><li><a href="#Compression_and_packaging_formats">Compression and packaging formats</a></li><li><a href="#Text_formats">Text formats</a></li><li><a href="#Audio_formats">Audio formats</a></li><li><a h
ref="#Image_formats">Image formats</a></li><li><a href="#Video_formats">Video formats</a></li><li><a href="#Java_class_files_and_archives">Java class files and archives</a></li><li><a href="#The_mbox_format">The mbox format</a></li></ul></li></ul><div class="section"><h3><a name="HyperText_Markup_Language">HyperText Markup Language</a><a name="HyperText_Markup_Language"></a></h3><p>The HyperText Markup Language (HTML) is the lingua franca of the web. Tika uses the <a class="externalLink" href="http://home.ccil.org/~cowan/XML/tagsoup/">TagSoup</a> library to support virtually any kind of HTML found on the web. The output from the <a href="./api/org/apache/tika/parser/html/HtmlParser.html">HtmlParser</a> class is guaranteed to be well-formed and valid XHTML, and various heuristics are used to prevent things like inline scripts from cluttering the extracted text content.</p></div><div class="section"><h3><a name="XML_and_derived_formats">XML and derived formats</a><a name="XML_
and_derived_formats"></a></h3><p>The Extensible Markup Language (XML) format is a generic format that can be used for all kinds of content. Tika has custom parsers for some widely used XML vocabularies like XHTML, OOXML and ODF, but the default <a href="./api/org/apache/tika/parser/xml/DcXMLParser.html">DcXMLParser</a> class simply extracts the text content of the document and ignores any XML structure. The only exception to this rule are Dublin Core metadata elements that are used for the document metadata.</p></div><div class="section"><h3><a name="Microsoft_Office_document_formats">Microsoft Office document formats</a><a name="Microsoft_Office_document_formats"></a></h3><p>Microsoft Office and some related applications produce documents in the generic OLE 2 Compound Document and Office Open XML (OOXML) formats. The older OLE 2 format was introduced in Microsoft Office version 97 and was the default format until Office version 2007 and the new XML-based OOXML format. The <
a href="./api/org/apache/tika/parser/microsoft/OfficeParser.html">OfficeParser</a> and <a href="./api/org/apache/tika/parser/microsoft/ooxml/OOXMLParser.html">OOXMLParser</a> classes use <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> libraries to support text and metadata extraction from both OLE2 and OOXML documents.</p></div><div class="section"><h3><a name="OpenDocument_Format">OpenDocument Format</a><a name="OpenDocument_Format"></a></h3><p>The OpenDocument format (ODF) is used most notably as the default format of the OpenOffice.org office suite. The <a href="./api/org/apache/tika/parser/odf/OpenDocumentParser.html">OpenDocumentParser</a> class supports this format and the earlier OpenOffice 1.0 format on which ODF is based.</p></div><div class="section"><h3><a name="Portable_Document_Format">Portable Document Format</a><a name="Portable_Document_Format"></a></h3><p>The <a href="./api/org/apache/tika/parser/pdf/PDFParser.html">PDFParser</a> class p
arsers Portable Document Format (PDF) documents using the <a class="externalLink" href="http://pdfbox.apache.org/">Apache PDFBox</a> library.</p></div><div class="section"><h3><a name="Electronic_Publication_Format">Electronic Publication Format</a><a name="Electronic_Publication_Format"></a></h3><p>The <a href="./api/org/apache/tika/parser/epub/EpubParser.html">EpubParser</a> class supports the Electronic Publication Format (EPUB) used for many digital books.</p></div><div class="section"><h3><a name="Rich_Text_Format">Rich Text Format</a><a name="Rich_Text_Format"></a></h3><p>The <a href="./api/org/apache/tika/parser/rtf/RTFParser.html">RTFParser</a> class uses the standard javax.swing.text.rtf feature to extract text content from Rich Text Format (RTF) documents.</p></div><div class="section"><h3><a name="Compression_and_packaging_formats">Compression and packaging formats</a><a name="Compression_and_packaging_formats"></a></h3><p>Tika uses the <a class="externalLink" hre
f="http://commons.apache.org/compress/">Commons Compress</a> library to support various compression and packaging formats. The <a href="./api/org/apache/tika/parser/pkg/PackageParser.html">PackageParser</a> class and its subclasses first parse the top level compression or packaging format and then pass the unpacked document streams to a second parsing stage using the parser instance specified in the parse context.</p></div><div class="section"><h3><a name="Text_formats">Text formats</a><a name="Text_formats"></a></h3><p>Extracting text content from plain text files seems like a simple task until you start thinking of all the possible character encodings. The <a href="./api/org/apache/tika/parser/txt/TXTParser.html">TXTParser</a> class uses encoding detection code from the <a class="externalLink" href="http://site.icu-project.org/">ICU</a> project to automatically detect the character encoding of a text document.</p></div><div class="section"><h3><a name="Audio_formats">Audio
formats</a><a name="Audio_formats"></a></h3><p>Tika can detect several common audio formats and extract metadata from them. Even text extraction is supported for some audio files that contain lyrics or other textual content. The <a href="./api/org/apache/tika/parser/audio/AudioParser.html">AudioParser</a> and <a href="./api/org/apache/tika/parser/audio/MidiParser.html">MidiParser</a> classes use standard javax.sound features to process simple audio formats, and the <a href="./api/org/apache/tika/parser/mp3/Mp3Parser.html">Mp3Parser</a> class adds support for the widely used MP3 format.</p></div><div class="section"><h3><a name="Image_formats">Image formats</a><a name="Image_formats"></a></h3><p>The <a href="./api/org/apache/tika/parser/image/ImageParser.html">ImageParser</a> class uses the standard javax.imageio feature to extract simple metadata from image formats supported by the Java platform. More complex image metadata is available through the <a href="./api/org/apache
/tika/parser/jpeg/JpegParser.html">JpegParser</a> class that uses the metadata-extractor library to supports Exif metadata extraction from Jpeg images.</p></div><div class="section"><h3><a name="Video_formats">Video formats</a><a name="Video_formats"></a></h3><p>Currently Tika only supports the Flash video format using a simple parsing algorithm implemented in the <a href="./api/org/apache/tika/parser/flv/FLVParser">FLVParser</a> class.</p></div><div class="section"><h3><a name="Java_class_files_and_archives">Java class files and archives</a><a name="Java_class_files_and_archives"></a></h3><p>The <a href="./api/org/apache/tika/parser/asm/ClassParser">ClassParser</a> class extracts class names and method signatures from Java class files, and the <a href="./api/org/apache/tika/parser/pkg/ZipParser.html">ZipParser</a> class supports also jar archives.</p></div><div class="section"><h3><a name="The_mbox_format">The mbox format</a><a name="The_mbox_format"></a></h3><p>The <a href
="./api/org/apache/tika/parser/mbox/MboxParser.html">MboxParser</a> can extract email messages from the mbox format used by many email archives and Unix-style mailboxes.</p></div></div>
</div>
</div>
<div class="clear">
Modified: tika/site/publish/0.6/parser.html
URL: http://svn.apache.org/viewvc/tika/site/publish/0.6/parser.html?rev=982632&r1=982631&r2=982632&view=diff
==============================================================================
--- tika/site/publish/0.6/parser.html (original)
+++ tika/site/publish/0.6/parser.html Thu Aug 5 14:19:17 2010
@@ -192,9 +192,9 @@
</div>
<div id="bodyColumn">
<div id="contentBox">
- <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>The Parser interface<a name="The_Parser_interface"></a></h2><p>The <a href="#apiorgapachetika
parserParser.html">org.apache.tika.parser.Parser</a> interface is the key concept of Apache Tika. It hides the complexity of different file formats and parsing libraries while providing a simple and powerful mechanism for client applications to extract structured text content and metadata from all sorts of documents. All this is achieved with a single method:</p><div><pre>void parse(
+ <!-- Licensed to the Apache Software Foundation (ASF) under one or more --><!-- contributor license agreements. See the NOTICE file distributed with --><!-- this work for additional information regarding copyright ownership. --><!-- The ASF licenses this file to You under the Apache License, Version 2.0 --><!-- (the "License"); you may not use this file except in compliance with --><!-- the License. You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. --><div class="section"><h2>The Parser interface<a name="The_Parser_interface"></a></h2><p>The <a href="./api/org/apache/
tika/parser/Parser.html">org.apache.tika.parser.Parser</a> interface is the key concept of Apache Tika. It hides the complexity of different file formats and parsing libraries while providing a simple and powerful mechanism for client applications to extract structured text content and metadata from all sorts of documents. All this is achieved with a single method:</p><div><pre>void parse(
InputStream stream, ContentHandler handler, Metadata metadata,
- ParseContext context) throws IOException, SAXException, TikaException;</pre></div><p>The <tt>parse</tt> method takes the document to be parsed and related metadata as input and outputs the results as XHTML SAX events and extra metadata. The parse context argument is used to specify context information (like the current local) that is not related to any individual document. The main criteria that lead to this design were:</p><dl><dt>Streamed parsing</dt><dd>The interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. This allows even huge documents to be parsed without excessive resource requirements.</dd><dt>Structured content</dt><dd>A parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. A client application can use this information for example to better judge the relevance of different parts of the parsed docum
ent.</dd><dt>Input metadata</dt><dd>A client application should be able to include metadata like the file name or declared content type with the document to be parsed. The parser implementation can use this information to better guide the parsing process.</dd><dt>Output metadata</dt><dd>A parser implementation should be able to return document metadata in addition to document content. Many document formats contain metadata like the name of the author that may be useful to client applications.</dd><dt>Context sensitivity</dt><dd>While the default settings and behaviour of Tika parsers should work well for most use cases, there are still situations where more fine-grained control over the parsing process is desirable. It should be easy to inject such context-specific information to the parsing process without breaking the layers of abstraction.</dd></dl><p>These criteria are reflected in the arguments of the <tt>parse</tt> method.</p><div class="section"><h3>Document input str
eam<a name="Document_input_stream"></a></h3><p>The first argument is an <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html">InputStream</a> for reading the document to be parsed.</p><p>If this document stream can not be read, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html">IOException</a> is passed up to the client application. If the stream can be read but not parsed (for example if the document is corrupted), then the parser throws a <a href="#apiorgapachetikaexceptionTikaException.html">TikaException</a>.</p><p>The parser implementation will consume this stream but <i>will not close it</i>. Closing the stream is the responsibility of the client application that opened it in the first place. The recommended pattern for using streams with the <tt>parse</tt> method is:</p><div><pre>InputStream stream = ...; // open the stream
+ ParseContext context) throws IOException, SAXException, TikaException;</pre></div><p>The <tt>parse</tt> method takes the document to be parsed and related metadata as input and outputs the results as XHTML SAX events and extra metadata. The parse context argument is used to specify context information (like the current local) that is not related to any individual document. The main criteria that lead to this design were:</p><dl><dt>Streamed parsing</dt><dd>The interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. This allows even huge documents to be parsed without excessive resource requirements.</dd><dt>Structured content</dt><dd>A parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. A client application can use this information for example to better judge the relevance of different parts of the parsed docum
ent.</dd><dt>Input metadata</dt><dd>A client application should be able to include metadata like the file name or declared content type with the document to be parsed. The parser implementation can use this information to better guide the parsing process.</dd><dt>Output metadata</dt><dd>A parser implementation should be able to return document metadata in addition to document content. Many document formats contain metadata like the name of the author that may be useful to client applications.</dd><dt>Context sensitivity</dt><dd>While the default settings and behaviour of Tika parsers should work well for most use cases, there are still situations where more fine-grained control over the parsing process is desirable. It should be easy to inject such context-specific information to the parsing process without breaking the layers of abstraction.</dd></dl><p>These criteria are reflected in the arguments of the <tt>parse</tt> method.</p><div class="section"><h3>Document input str
eam<a name="Document_input_stream"></a></h3><p>The first argument is an <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/InputStream.html">InputStream</a> for reading the document to be parsed.</p><p>If this document stream can not be read, then parsing stops and the thrown <a class="externalLink" href="http://java.sun.com/j2se/1.5.0/docs/api/java/io/IOException.html">IOException</a> is passed up to the client application. If the stream can be read but not parsed (for example if the document is corrupted), then the parser throws a <a href="./api/org/apache/tika/exception/TikaException.html">TikaException</a>.</p><p>The parser implementation will consume this stream but <i>will not close it</i>. Closing the stream is the responsibility of the client application that opened it in the first place. The recommended pattern for using streams with the <tt>parse</tt> method is:</p><div><pre>InputStream stream = ...; // open the stream
try {
parser.parse(stream, ...); // parse the stream
} finally {
@@ -206,14 +206,14 @@ try {
<body>
...
</body>
-</html></pre></div><p>Parser implementations typically use the <a href="#apidocsorgapachetikasaxXHTMLContentHandler.html">XHTMLContentHandler</a> utility class to generate the XHTML output.</p><p>Dealing with the raw SAX events can be a bit complex, so Apache Tika comes with a number of utility classes that can be used to process and convert the event stream to other representations.</p><p>For example, the <a href="#apiorgapachetikasaxBodyContentHandler.html">BodyContentHandler</a> class can be used to extract just the body part of the XHTML output and feed it either as SAX events to another content handler or as characters to an output stream, a writer, or simply a string. The following code snippet parses a document from the standard input stream and outputs the extracted text content to standard output:</p><div><pre>ContentHandler handler = new BodyContentHandler(System.out);
-parser.parse(System.in, handler, ...);</pre></div><p>Another useful class is <a href="#apiorgapachetikaparserParsingReader.html">ParsingReader</a> that uses a background thread to parse the document and returns the extracted text content as a character stream:</p><div><pre>InputStream stream = ...; // the document to be parsed
+</html></pre></div><p>Parser implementations typically use the <a href="./api/org/apache/tika/sax/XHTMLContentHandler.html">XHTMLContentHandler</a> utility class to generate the XHTML output.</p><p>Dealing with the raw SAX events can be a bit complex, so Apache Tika comes with a number of utility classes that can be used to process and convert the event stream to other representations.</p><p>For example, the <a href="./api/org/apache/tika/sax/BodyContentHandler.html">BodyContentHandler</a> class can be used to extract just the body part of the XHTML output and feed it either as SAX events to another content handler or as characters to an output stream, a writer, or simply a string. The following code snippet parses a document from the standard input stream and outputs the extracted text content to standard output:</p><div><pre>ContentHandler handler = new BodyContentHandler(System.out);
+parser.parse(System.in, handler, ...);</pre></div><p>Another useful class is <a href="./api/org/apache/tika/parser/ParsingReader.html">ParsingReader</a> that uses a background thread to parse the document and returns the extracted text content as a character stream:</p><div><pre>InputStream stream = ...; // the document to be parsed
Reader reader = new ParsingReader(parser, stream, ...);
try {
...; // read the document text using the reader
} finally {
reader.close(); // the document stream is closed automatically
-}</pre></div></div><div class="section"><h3>Document metadata<a name="Document_metadata"></a></h3><p>The third argument to the <tt>parse</tt> method is used to pass document metadata both in and out of the parser. Document metadata is expressed as an <a href="#apiorgapachetikametadataMetadata.html">Metadata</a> object.</p><p>The following are some of the more interesting metadata properties:</p><dl><dt>Metadata.RESOURCE_NAME_KEY</dt><dd>The name of the file or resource that contains the document.<p>A client application can set this property to allow the parser to use file name heuristics to determine the format of the document.</p><p>The parser implementation may set this property if the file format contains the canonical name of the file (for example the Gzip format has a slot for the file name).</p></dd><dt>Metadata.CONTENT_TYPE</dt><dd>The declared content type of the document.<p>A client application can set this property based on for example a HTTP Content-Type header. T
he declared content type may help the parser to correctly interpret the document.</p><p>The parser implementation sets this property to the content type according to which the document was parsed.</p></dd><dt>Metadata.TITLE</dt><dd>The title of the document.<p>The parser implementation sets this property if the document format contains an explicit title field.</p></dd><dt>Metadata.AUTHOR</dt><dd>The name of the author of the document.<p>The parser implementation sets this property if the document format contains an explicit author field.</p></dd></dl><p>Note that metadata handling is still being discussed by the Tika development team, and it is likely that there will be some (backwards incompatible) changes in metadata handling before Tika 1.0.</p></div><div class="section"><h3>Parse context<a name="Parse_context"></a></h3><p>The final argument to the <tt>parse</tt> method is used to inject context-specific information to the parsing process. This is useful for example when
dealing with locale-specific date and number formats in Microsoft Excel spreadsheets. Another important use of the parse context is passing in the delegate parser instance to be used by two-phase parsers like the <a href="#apiorgapacheparserpkgPackageParser.html">PackageParser</a> subclasses. Some parser classes allow customization of the parsing process through strategy objects in the parse context.</p></div><div class="section"><h3>Parser implementations<a name="Parser_implementations"></a></h3><p>Apache Tika comes with a number of parser classes for parsing <a href="#formats.html">various document formats</a>. You can also extend Tika with your own parsers, and of course any contributions to Tika are warmly welcome.</p><p>The goal of Tika is to reuse existing parser libraries like <a class="externalLink" href="http://www.pdfbox.org/">PDFBox</a> or <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> as much as possible, and so most of the parser classes in
Tika are adapters to such external libraries.</p><p>Tika also contains some general purpose parser implementations that are not targeted at any specific document formats. The most notable of these is the <a href="#apidocsorgapachetikaparserAutoDetectParser.html">AutoDetectParser</a> class that encapsulates all Tika functionality into a single parser that can handle any types of documents. This parser will automatically determine the type of the incoming document based on various heuristics and will then parse the document accordingly.</p></div></div>
+}</pre></div></div><div class="section"><h3>Document metadata<a name="Document_metadata"></a></h3><p>The third argument to the <tt>parse</tt> method is used to pass document metadata both in and out of the parser. Document metadata is expressed as an <a href="./api/org/apache/tika/metadata/Metadata.html">Metadata</a> object.</p><p>The following are some of the more interesting metadata properties:</p><dl><dt>Metadata.RESOURCE_NAME_KEY</dt><dd>The name of the file or resource that contains the document.<p>A client application can set this property to allow the parser to use file name heuristics to determine the format of the document.</p><p>The parser implementation may set this property if the file format contains the canonical name of the file (for example the Gzip format has a slot for the file name).</p></dd><dt>Metadata.CONTENT_TYPE</dt><dd>The declared content type of the document.<p>A client application can set this property based on for example a HTTP Content-Type hea
der. The declared content type may help the parser to correctly interpret the document.</p><p>The parser implementation sets this property to the content type according to which the document was parsed.</p></dd><dt>Metadata.TITLE</dt><dd>The title of the document.<p>The parser implementation sets this property if the document format contains an explicit title field.</p></dd><dt>Metadata.AUTHOR</dt><dd>The name of the author of the document.<p>The parser implementation sets this property if the document format contains an explicit author field.</p></dd></dl><p>Note that metadata handling is still being discussed by the Tika development team, and it is likely that there will be some (backwards incompatible) changes in metadata handling before Tika 1.0.</p></div><div class="section"><h3>Parse context<a name="Parse_context"></a></h3><p>The final argument to the <tt>parse</tt> method is used to inject context-specific information to the parsing process. This is useful for example
when dealing with locale-specific date and number formats in Microsoft Excel spreadsheets. Another important use of the parse context is passing in the delegate parser instance to be used by two-phase parsers like the <a href="./api/org/apache/parser/pkg/PackageParser.html">PackageParser</a> subclasses. Some parser classes allow customization of the parsing process through strategy objects in the parse context.</p></div><div class="section"><h3>Parser implementations<a name="Parser_implementations"></a></h3><p>Apache Tika comes with a number of parser classes for parsing <a href="./formats.html">various document formats</a>. You can also extend Tika with your own parsers, and of course any contributions to Tika are warmly welcome.</p><p>The goal of Tika is to reuse existing parser libraries like <a class="externalLink" href="http://www.pdfbox.org/">PDFBox</a> or <a class="externalLink" href="http://poi.apache.org/">Apache POI</a> as much as possible, and so most of the pars
er classes in Tika are adapters to such external libraries.</p><p>Tika also contains some general purpose parser implementations that are not targeted at any specific document formats. The most notable of these is the <a href="./api/org/apache/tika/parser/AutoDetectParser.html">AutoDetectParser</a> class that encapsulates all Tika functionality into a single parser that can handle any types of documents. This parser will automatically determine the type of the incoming document based on various heuristics and will then parse the document accordingly.</p></div></div>
</div>
</div>
<div class="clear">