You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by Jukka Zitting <ju...@gmail.com> on 2008/12/14 22:54:15 UTC
Dropping or repurposing the CHANGES file
Hi,
We use the CHANGES.txt file to record all the issues that have been
fixed in the codebase. However, the same information (and much more)
is already conveniently accessible in Jira, see for example [1] for
the 0.2 release. Would anyone mind if we just dropped the CHANGES.txt
file?
Alternatively we could repurpose it as a higher level release notes
file that records more substantial changes in more detail. For example
instead of just one-liners with just the bug titles, we could for
example include a paragraph or so for each notable API or
configuration change and just refer to Jira for the less notable
changes.
[1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310631&styleName=Html&version=12312902
BR,
Jukka Zitting
Re: Dropping or repurposing the CHANGES file
Posted by Grant Ingersoll <gs...@apache.org>.
On Dec 16, 2008, at 5:46 AM, Jukka Zitting wrote:
> Hi,
>
> On Mon, Dec 15, 2008 at 8:26 PM, Mattmann, Chris A
> <ch...@jpl.nasa.gov> wrote:
>> +1, I agree with Grant's comment. I also like the simplicity of
>> having a
>> text file, separate of JIRA. What I find quite often (we use JIRA
>> at work,
>> and deploy our open source software on various projects) is that
>> folks don't
>> know what JIRA is, or care, but they like having a text file that
>> comes
>> along with the release that they can refer to, and at least get a
>> high level
>> view of what's going on in terms of updates and features for a
>> release.
>
> Agreed on the importance of having high level release notes in
> writing. However, I don't think that the current CHANGES.txt format of
> just listing the resolved issues is any better than the report from
> Jira.
>
>> Sure, JIRA can generate that file for you, using the release notes,
>> but like
>> Grant said, it takes out the (some guy X via some committer Y), or
>> (some
>> committer Y) comments, which are nice to give folks credit.
>
> Jira also has a pretty good contribution report (see [1] for the 0.2
> release) where you can see all the people who've contributed to a
> release.
Very cool. I didn't know that. I agree that manually maintaining it
is a pain in the butt. Perhaps it would be reasonable to generate a
PDF from it for inclusion in a release, that way there is a more
permanent record of it included. In fact, that report is looks better
than the current way, which is usually left to the committer.
> [1] https://issues.apache.org/jira/secure/ConfigureReport.jspa?versionId=12312902&selectedProjectId=12310631&reportKey=com.sourcelabs.jira.plugin.report.contributions%3Acontributionreport&Next=Next
-Grant
Re: Dropping or repurposing the CHANGES file
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Tue, Dec 16, 2008 at 11:46 AM, Jukka Zitting <ju...@gmail.com> wrote:
> See the end of this message for a quick draft of what a higher level
> changelog could look like (using 0.2 as the example). I would much
> rather see us producing such a document than just mechanically listing
> fixed issues in the current CHANGES.txt.
I took the liberty of updating the change log along these lines for
the upcoming 0.3 release. See below for what the current CHANGES.txt
file looks like.
The idea is that the change log would be a document that people would
actually want to read to see what has been changed in each new
release. It would contain summaries of only the most notable changes
and just point to Jira for all the details and the less important
changes.
BR,
Jukka Zitting
Apache Tika Change Log
======================
Unreleased changes (0.3)
------------------------
The most notable changes in Tika 0.3 over the previous release are:
* Tika now disables the loading of all external entities in XML files
that it parses as input documents. This improves security and avoids
problems with potentially broken references. (TIKA-185)
* Tika now replaces all invalid XML characters in the extracted text
content with spaces. This prevents problems when output from Tika
is processed with XML tools. (TIKA-180)
* The Tika CLI now correctly flushes its buffers when invoked with the
--text argument. This prevents the end of the text output from being
lost. (TIKA-179)
See http://tinyurl.com/tika-0-3-changes for a list of all changes in Tika 0.3.
The following people have contributed to Tika 0.3 by submitting or commenting
on the issues resolved in this release:
Andrzej Rusin
Chris A. Mattmann
Dave Meikle
Guillermo Arribas
Jukka Zitting
Paul Borgermans
Peter Becker
Sébastien Michel
Uwe Schindler
See http://tinyurl.com/tika-0-3-contributions for more details on
these contributions.
Release 0.2 - 12/04/2008
------------------------
1. TIKA-109 - WordParser fails on some Word files (Dave Meikle)
2. TIKA-105 - Excel parser implementation based on POI's Event API
(Niall Pemberton)
3. TIKA-116 - Streaming parser for OpenDocument files (Jukka Zitting)
4. TIKA-117 - Drop JDOM and Jaxen dependencies (Jukka Zitting)
5. TIKA-115 - Tika package with all the dependencies (Jukka Zitting)
6. TIKA-97 - Tika GUI (Jukka Zitting)
7. TIKA-96 - Tika CLI (Jukka Zitting)
8. TIKA-112 - Use Commons IO 1.4 (Jukka Zitting)
9. TIKA-127 - Add support for Visio files (Jukka Zitting)
10. TIKA-129 - node() support for the streaming XPath utility (Jukka Zitting)
11. TIKA-130 - self-or-descendant axis does not match self in streaming XPath
(Jukka Zitting)
12. TIKA-131 - Lazy XHTML prefix generation (Jukka Zitting)
13. TIKA-128 - HTML parser should produce XHTML SAX events (Jukka Zitting)
14. TIKA-133 - TeeContentHandler constructor should use varargs (Jukka Zitting)
15. TIKA-132 - Refactor Excel extractor to parse per sheet and add
hyperlink support (Niall Pemberton)
16. TIKA-134 - mvn package does not produce packages for bin/src
(Karl Heinz Marbaise)
17. TIKA-138 - Ignore HTML style and script content (Jukka Zitting)
18. TIKA-113 - Metadata (such as title) should not be part of content
(Jukka Zitting)
19. TIKA-139 - Add a composite parser (Jukka Zitting)
20. TIKA-142 - Include application/xhtml+xml as valid mime type for XMLParser
(mattmann)
21. TIKA-143 - Add ParsingReader (Jukka Zitting)
22. TIKA-144 - Upgrade nekohtml dependency (Jukka Zitting)
23. TIKA-145 - Separate NOTICEs and LICENSEs for binary and source packages
(Jukka Zitting)
24. TIKA-146 - Upgrade to POI 3.1 (Jukka Zitting)
25. TIKA-99 - Support external parser programs (Jukka Zitting)
26. TIKA-149 - Parser for Zip files (Dave Meikle & Jukka Zitting)
27. TIKA-150 - Parser for tar files (Jukka Zitting)
28. TIKA-151 - Stream compression support (Jukka Zitting)
29. TIKA-156 - Some MIME magic patterns are ignored by MimeTypes
(Jukka Zitting)
30. TIKA-155 - Java class file parser (Dave Brosius & Jukka Zitting)
31. TIKA-108 - New Tika logos (Yongqian Li & Jukka Zitting)
32. TIKA-120 - Add support for retrieving ID3 tags from MP3 files
(Dave Meikle & Jukka Zitting)
33. TIKA-54 - Outlook msg parser
(Rida Benjelloun, Dave Meikle & Jukka Zitting)
34. TIKA-114 - PDFParser : Getting content of the document using
"writer.ToString ()" , some words are stuck together
(Dave Meikle)
35. TIKA-161 - Enable PMD reports (Jukka Zitting)
36. TIKA-159 - Add support for parsing basic audio types: wav, aiff, au, midi
(Sami Siren)
37. TIKA-140 - HTML parser unable to extract text
(Julien Nioche & Jukka Zitting)
38. TIKA-163 - GUI does not support drag and drop in Gnome or KDE (Dave Meikle)
39. TIKA-166 - Update HTMLParser to parse contents of meta tags (Dave Meikle)
40. TIKA-164 - Upgrade of the nekohtml dependency to 1.9.9 (Jukka Zitting)
41. TIKA-165 - Upgrade of the ICU4J dependency to version 3.8 (Jukka Zitting)
42. TIKA-172 - New Open Document Parser that emits structured XHTML content
(Uwe Schindler & Jukka Zitting)
43. TIKA-175 - Retrotranslate Tika for use in Java 1.4 environments
(Jukka Zitting)
44. TIKA-177 - Improvements to build instruction in README (Chris
Hostetter & Jukka Zitting)
45. TIKA-171 - New ContentHandler for plain text output that has no problem with
missing white space after XHTML block tags (Uwe
Schindler & Jukka Zitting)
Release 0.1-incubating - 12/27/2007
-----------------------------------
1. TIKA-5 - Port Metadata Framework from Nutch (mattmann)
2. TIKA-11 - Consolidate test classes into a src/test/java directory
tree (mattmann)
3. TIKA-15 - Utils.print does not print a Content having no value (jukka)
4. TIKA-19 - org.apache.tika.TestParsers fails (bdelacretaz)
5. TIKA-16 - Issues with data files used for testing by TestParsers
(bdelacretaz)
6. TIKA-14 - MimeTypeUtils.getMimeType() returns the default mime type for
.odt (Open Office) file (bdelacretaz)
7. TIKA-12 - Add URL capability to MimeTypesUtils (jukka)
8. TIKA-13 - Fix obsolete package names in config.xml (siren)
9. TIKA-10 - Remove MimeInfoException catch clauses and import from
TestParsers (siren)
10. TIKA-8 - Replaced the jmimeinfo dependency with a trivial mime
type detector (jukka)
11. TIKA-7 - Added the Lius Lite code. Added missing dependencies to POM (jukka)
12. TIKA-18 - "Office" interface should be renamed "MSOffice" (mattmann)
13. TIKA-23 - Decouple Parser from ParserConfig (jukka)
14. TIKA-6 - Port Nutch (or better) MimeType detection system into
Tika (J. Charron & mattmann)
15. TIKA-25 - Removed hardcoded reference to C:\oo.xml in
OpenOfficeParser (K. Bennett & jukka)
16. TIKA-17 - Need to support URL's for input resources. (K. Bennett & mattmann)
17. TIKA-22 - Remove @author tags from the java source (mattmann)
18. TIKA-21 - Simplified configuration code (jukka)
19. TIKA-17 - Rename all "Lius" classes to be "Tika" classes (jukka)
20. TIKA-30 - Added utility constructors to TikaConfig (K. Bennett & jukka)
21. TIKA-28 - Rename config.xml to tika-config.xml or similar (mattmann)
22. TIKA-26 - Use Map<String, Content> instead of List<Content> (jukka)
23. TIKA-31 - protected Parser.parse(InputStream stream,
Iterable<Content> contents) (jukka & K. Bennett)
24. TIKA-36 - A convenience method for getting a document's content's text
would be helpful (K. Bennett & mattmann)
25. TIKA-33 - Stateless parsers (jukka)
26. TIKA-38 - TXTParser adds a space to the content it reads from a
file (K. Bennett & ridabenjelloun)
27. TIKA-35 - Extract MsOffice properties, use RereadableInputStream
devloped by K. Bennett (ridabenjelloun & K. Bennett)
28. TIKA-39 - Excel parsing improvements (siren & ridabenjelloun)
29. TIKA-34 - Provide a method that will return a default configuration
(TikaConfig) (K. Bennett & mattmann)
30. TIKA-42 - Content class needs (String, String, String) constructor
(K. Bennett)
31. TIKA-43 - Parser interface (jukka)
32. TIKA-47 - Remove TikaLogger (jukka)
33. TIKA-46 - Use Metadata in Parser (jukka & mattmann)
34. TIKA-48 - Merge MS Extractors and Parsers (jukka)
35. TIKA-45 - RereadableInputStream needs to be able to read to
the end of the original stream on first rewind. (K. Bennett)
36. TIKA-41 - Resource files occur twice in jar file. (jukka)
37. TIKA-49 - Some files have old-style license headers, fixed (Robert
Burrell Donkin & bdelacretaz)
38. TIKA-51 - Leftover temp files after running Tika tests, fixed (bdelacretaz)
39. TIKA-40 - Tika needs to support diverse character encodings (jukka)
40. TIKA-55 - ParseUtils.getParser() method variants should have
consistent parameter orders
(K. Bennett)
41. TIKA-52 - RereadableInputStream needs to support not closing the
input stream it wraps.
(K. Bennett via bdelacretaz)
42. TIKA-53 - XHTML SAX events from parsers (jukka)
43. TIKA-57 - Rename org.apache.tika.ms to org.apache.tika.parser.ms (jukka)
44. TIKA-62 - Use TikaConfig.getDefaultConfig() instead of a hardcoded
config path in TestParsers (jukka)
45. TIKA-58 - Replace jtidy html parser with nekohtml based parser (siren)
46. TIKA-60 - Rename Microsoft parser classes (jukka)
47. TIKA-63 - Avoid multiple passes over the input stream in Microsoft parsers
(jukka)
48. TIKA-66 - Use Java 5 features in org.apache.tika.mime (jukka)
49. TIKA-56 - Mime type detection fails with upper case file
extensions such as "PDF"
(mattmann)
50. TIKA-65 - Add encode detection support for HTML parser (siren)
51. TIKA-68 - Add dummy parser classes to be used as sentinels (jukka)
52. TIKA-67 - Add an auto-detecting Parser implementation (jukka)
53. TIKA-70 - Better MIME information for the Open Document formats (jukka)
54. TIKA-71 - Remove ParserConfig and ParserFactory (jukka)
55. TIKA-83 - Create a org.apache.tika.sax package for SAX utilities (jukka)
56. TIKA-84 - Add MimeTypes.getMimeType(InputStream) (jukka)
57. TIKA-85 - Add glob patterns from the ASF svn:eol-style documentation (jukka)
58. TIKA-100 - Structured PDF parsing (jukka)
59. TIKA-101 - Improve site and build (mattmann)
60. TIKA-102 - Parser implementations loading a large amount of content
into a single String could be problematic (Niall Pemberton)
61. TIKA-107 - Remove use of assertions for argument checking (Niall Pemberton)
62. TIKA-104 - Add utility methods to throw IOException with the caused
intialized (jukka & Niall Pemberton)
63. TIKA-106 - Remove dependency on Jakarta ORO - use JDK 1.4 Regex
(Niall Pemberton)
64. TIKA-111 - Missing license headers (jukka)
65. TIKA-112 - XMLParser improvement (ridabenjelloun)
Re: Dropping or repurposing the CHANGES file
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Dec 15, 2008 at 8:26 PM, Mattmann, Chris A
<ch...@jpl.nasa.gov> wrote:
> +1, I agree with Grant's comment. I also like the simplicity of having a
> text file, separate of JIRA. What I find quite often (we use JIRA at work,
> and deploy our open source software on various projects) is that folks don't
> know what JIRA is, or care, but they like having a text file that comes
> along with the release that they can refer to, and at least get a high level
> view of what's going on in terms of updates and features for a release.
Agreed on the importance of having high level release notes in
writing. However, I don't think that the current CHANGES.txt format of
just listing the resolved issues is any better than the report from
Jira.
> Sure, JIRA can generate that file for you, using the release notes, but like
> Grant said, it takes out the (some guy X via some committer Y), or (some
> committer Y) comments, which are nice to give folks credit.
Jira also has a pretty good contribution report (see [1] for the 0.2
release) where you can see all the people who've contributed to a
release.
I'm not saying that we should use these reports as-is as the only
release notes, but since they already cover all the information we
have in the current CHANGES.txt file I don't see much point in
manually maintaining the file in it's current format.
See the end of this message for a quick draft of what a higher level
changelog could look like (using 0.2 as the example). I would much
rather see us producing such a document than just mechanically listing
fixed issues in the current CHANGES.txt.
[1] https://issues.apache.org/jira/secure/ConfigureReport.jspa?versionId=12312902&selectedProjectId=12310631&reportKey=com.sourcelabs.jira.plugin.report.contributions%3Acontributionreport&Next=Next
BR,
Jukka Zitting
----
What's new in Apache Tika 0.2
=============================
Apache Tika 0.2 is an incremental feature release. The most notable changes
in this release are:
* The Microsoft Excel parser now supports partial streaming (per sheet)
and makes cell hyperlinks available as XHTML <a/> tags.
* The HTML parser now produces proper XHTML SAX events, extracts <head/>
metadata better, and avoids including script or style information in
the extracted text content.
* The OpenDocument parser now supports streaming and structured content.
* New parsers for the following document formats:
- Microsoft Outlook and Visio documents
- MP3 (ID3), WAV, AIFF, AU and MIDI audio files
- zip, jar and tar archives
- Java .class files
* Tika now supports Gzip and Bzip2 decompression.
* The new ParsingReader class makes the extracted text content of a document
available as a java.io.Reader. This is especially useful for integration
with Lucene Java that uses the Reader class for reading text to be indexed.
* The new ExternalParser class allows external parser programs to be invoked
through the Tika Parser API.
* Tika now has simple command line and graphical user interfaces designed
for easy testing or scripting of parsing functionality.
* A retrotranslated version of Tika is now available for use in
Java 1.4 environments.
See http://tinyurl.com/tika-0-2-notes for more details on all resolved issues.
Credits
-------
The following people have contributed to this release by committing changes,
by submitting patches, or by commenting on the issues resolved in the release.
Dave Meikle
Jukka Zitting
Uwe Schindler
Julien Nioche
Niall Pemberton
Sami Siren
Rida Benjelloun
Chris Hostetter
Mats Norén
Chris A. Mattmann
Litrik De Roy
Bertrand Delacretaz
Karl Heinz Marbaise
See http://tinyurl.com/tika-0-2-credits for more details on the contributions.
Re: Dropping or repurposing the CHANGES file
Posted by "Mattmann, Chris A" <ch...@jpl.nasa.gov>.
+1, I agree with Grant's comment. I also like the simplicity of having a
text file, separate of JIRA. What I find quite often (we use JIRA at work,
and deploy our open source software on various projects) is that folks don't
know what JIRA is, or care, but they like having a text file that comes
along with the release that they can refer to, and at least get a high level
view of what's going on in terms of updates and features for a release.
Sure, JIRA can generate that file for you, using the release notes, but like
Grant said, it takes out the (some guy X via some committer Y), or (some
committer Y) comments, which are nice to give folks credit. The decision was
made a while back on Tika to remove @author tags from the source b/c of the
fact that credit was given in e.g., CHANGES.txt and JIRA...
My 2 cents,
Chris
On 12/15/08 11:19 AM, "Grant Ingersoll" <gs...@apache.org> wrote:
> We dropped it for Mahout and did like in the link below, but one thing
> I don't like about it is it gets harder to give credit to (and
> identify) those who chipped in, but maybe aren't listed as the
> Reporter/Assignee on the issue. For example, it is often the case
> that someone reports, someone else generates the patch, and yet
> another person commits it.
>
> On Dec 14, 2008, at 4:54 PM, Jukka Zitting wrote:
>
>> Hi,
>>
>> We use the CHANGES.txt file to record all the issues that have been
>> fixed in the codebase. However, the same information (and much more)
>> is already conveniently accessible in Jira, see for example [1] for
>> the 0.2 release. Would anyone mind if we just dropped the CHANGES.txt
>> file?
>>
>> Alternatively we could repurpose it as a higher level release notes
>> file that records more substantial changes in more detail. For example
>> instead of just one-liners with just the bug titles, we could for
>> example include a paragraph or so for each notable API or
>> configuration change and just refer to Jira for the less notable
>> changes.
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310631&sty
>> leName=Html&version=12312902
>>
>> BR,
>>
>> Jukka Zitting
>
>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Disclaimer: The opinions presented within are my own and do not reflect
those of either NASA, JPL, or the California Institute of Technology.
Re: Dropping or repurposing the CHANGES file
Posted by Grant Ingersoll <gs...@apache.org>.
We dropped it for Mahout and did like in the link below, but one thing
I don't like about it is it gets harder to give credit to (and
identify) those who chipped in, but maybe aren't listed as the
Reporter/Assignee on the issue. For example, it is often the case
that someone reports, someone else generates the patch, and yet
another person commits it.
On Dec 14, 2008, at 4:54 PM, Jukka Zitting wrote:
> Hi,
>
> We use the CHANGES.txt file to record all the issues that have been
> fixed in the codebase. However, the same information (and much more)
> is already conveniently accessible in Jira, see for example [1] for
> the 0.2 release. Would anyone mind if we just dropped the CHANGES.txt
> file?
>
> Alternatively we could repurpose it as a higher level release notes
> file that records more substantial changes in more detail. For example
> instead of just one-liners with just the bug titles, we could for
> example include a paragraph or so for each notable API or
> configuration change and just refer to Jira for the less notable
> changes.
>
> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310631&styleName=Html&version=12312902
>
> BR,
>
> Jukka Zitting