You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by Ann Burgess <an...@gmail.com> on 2014/06/05 00:21:16 UTC
Review Request 22246: New parser for Matlab .mat files
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
Review request for tika and Chris Mattmann.
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs
-----
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Hey Annie,
The diff went away and is attached as a file? Can you change it to
upload diff (not a file)? Thanks!
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Tyler Palsulich <tp...@gmail.com>
Reply-To: "dev@tika.apache.org" <de...@tika.apache.org>, Tyler Palsulich
<tp...@gmail.com>
Date: Saturday, June 7, 2014 10:06 AM
To: Chris Mattmann <ma...@apache.org>
Cc: tika <de...@tika.apache.org>, "Bryant, Ann C (398J-Affiliate)"
<an...@gmail.com>, Tyler Palsulich <tp...@gmail.com>
Subject: Re: Review Request 22246: New parser for Matlab .mat files
>
>-----------------------------------------------------------
>This is an automatically generated e-mail. To reply, visit:
>https://reviews.apache.org/r/22246/#review45019
>-----------------------------------------------------------
>
>
>One more step for making the diff -- you have to run `svn add [file]` on
>all of the new files (the Parser and ParserTest). That way, they will be
>incorporated into the diff. When you run `svn status`, files with a '?'
>entry will not be in the diff, but ones with an 'A' will (along with the
>already-existing-but-modified 'M' files).
>
>- Tyler Palsulich
>
>
>On June 6, 2014, 11:07 p.m., Ann Burgess wrote:
>>
>> -----------------------------------------------------------
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/22246/
>> -----------------------------------------------------------
>>
>> (Updated June 6, 2014, 11:07 p.m.)
>>
>>
>> Review request for tika and Chris Mattmann.
>>
>>
>> Repository: tika
>>
>>
>> Description
>> -------
>>
>> This is a new parser for Matlab .mat files. The parser utilizes the
>>JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through
>>Maven Central. The text output from this parser provides variable names
>>and dimensions that are both inside and outside of data structures, but
>>does NOT provide the actual data values within each .mat file.
>>
>>
>> Diffs
>> -----
>>
>>
>> Diff: https://reviews.apache.org/r/22246/diff/
>>
>>
>> Testing
>> -------
>>
>> Successfully run a basic unit test that checks both --text and
>>--metadata parser output.
>>
>>
>> File Attachments
>> ----------------
>>
>> TIKA-1327.aburgess.140606.patch.txt
>>
>>https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-
>>4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
>>
>>
>> Thanks,
>>
>> Ann Burgess
>>
>>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Tyler Palsulich <tp...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review45019
-----------------------------------------------------------
One more step for making the diff -- you have to run `svn add [file]` on all of the new files (the Parser and ParserTest). That way, they will be incorporated into the diff. When you run `svn status`, files with a '?' entry will not be in the diff, but ones with an 'A' will (along with the already-existing-but-modified 'M' files).
- Tyler Palsulich
On June 6, 2014, 11:07 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 6, 2014, 11:07 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> TIKA-1327.aburgess.140606.patch.txt
> https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Chris Mattmann <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review45156
-----------------------------------------------------------
Annie, this looks great. I will just take out those 2 parts about the dependencies and then I think this is good to commit.
- Chris Mattmann
On June 9, 2014, 8:11 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 9, 2014, 8:11 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
> trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml 1601492
> trunk/tika-parsers/pom.xml 1601492
> trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
> trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java PRE-CREATION
> trunk/tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat UNKNOWN
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> TIKA-1327.aburgess.140606.patch.txt
> https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Tyler Palsulich <tp...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review45126
-----------------------------------------------------------
trunk/tika-parsers/pom.xml
<https://reviews.apache.org/r/22246/#comment79787>
Leave this as 4.2.20?
trunk/tika-parsers/pom.xml
<https://reviews.apache.org/r/22246/#comment79788>
Same -- leave this as 1.6.1?
trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java
<https://reviews.apache.org/r/22246/#comment79796>
Weird whitespace here. Try converting all tabs to 4 spaces.
- Tyler Palsulich
On June 9, 2014, 8:11 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 9, 2014, 8:11 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
> trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml 1601492
> trunk/tika-parsers/pom.xml 1601492
> trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
> trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java PRE-CREATION
> trunk/tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat UNKNOWN
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> TIKA-1327.aburgess.140606.patch.txt
> https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 13, 2014, 1:24 a.m.)
Review request for tika and Chris Mattmann.
Bugs: tika-1327
https://issues.apache.org/jira/browse/tika-1327
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs (updated)
-----
trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments
----------------
.mat test file
https://reviews.apache.org/media/uploaded/files/2014/06/10/43092452-6890-42cc-8254-fcbb1c8e07c6__breidamerkurjokull_radar_profiles_2009.mat
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 13, 2014, 1:23 a.m.)
Review request for tika and Chris Mattmann.
Bugs: tika-1327
https://issues.apache.org/jira/browse/tika-1327
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs (updated)
-----
trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments (updated)
----------------
.mat test file
https://reviews.apache.org/media/uploaded/files/2014/06/10/43092452-6890-42cc-8254-fcbb1c8e07c6__breidamerkurjokull_radar_profiles_2009.mat
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 10, 2014, 6:17 p.m.)
Review request for tika and Chris Mattmann.
Changes
-------
.mat file used for unit test.
Bugs: tika-1327
https://issues.apache.org/jira/browse/tika-1327
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs
-----
trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml 1601492
trunk/tika-parsers/pom.xml 1601492
trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java PRE-CREATION
trunk/tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat UNKNOWN
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments (updated)
----------------
TIKA-1327.aburgess.140606.patch.txt
https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
.mat test file
https://reviews.apache.org/media/uploaded/files/2014/06/10/43092452-6890-42cc-8254-fcbb1c8e07c6__breidamerkurjokull_radar_profiles_2009.mat
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 10, 2014, 3:21 a.m.)
Review request for tika and Chris Mattmann.
Bugs: tika-1327
https://issues.apache.org/jira/browse/tika-1327
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs
-----
trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml 1601492
trunk/tika-parsers/pom.xml 1601492
trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java PRE-CREATION
trunk/tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat UNKNOWN
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments
----------------
TIKA-1327.aburgess.140606.patch.txt
https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Chris Mattmann <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review45154
-----------------------------------------------------------
Ship it!
Ship It!
trunk/tika-parsers/pom.xml
<https://reviews.apache.org/r/22246/#comment79840>
agreed, this seems to be extraneous. I would remove this part.
trunk/tika-parsers/pom.xml
<https://reviews.apache.org/r/22246/#comment79842>
seems to be extraneous.
- Chris Mattmann
On June 9, 2014, 8:11 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 9, 2014, 8:11 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
> trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml 1601492
> trunk/tika-parsers/pom.xml 1601492
> trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
> trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java PRE-CREATION
> trunk/tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat UNKNOWN
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> TIKA-1327.aburgess.140606.patch.txt
> https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 9, 2014, 8:11 p.m.)
Review request for tika and Chris Mattmann.
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs (updated)
-----
trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml 1601492
trunk/tika-parsers/pom.xml 1601492
trunk/tika-parsers/src/main/java/org/apache/tika/parser/mat/MatParser.java PRE-CREATION
trunk/tika-parsers/src/test/java/org/apache/tika/parser/mat/MatParserTest.java PRE-CREATION
trunk/tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat UNKNOWN
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments
----------------
TIKA-1327.aburgess.140606.patch.txt
https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44980
-----------------------------------------------------------
File Attachment: TIKA-1327.aburgess.140606.patch.txt - TIKA-1327.aburgess.140606.patch.txt
<https://reviews.apache.org//r/22246/#fcomment21>
asc-227-180:tika IGSWAHWSWBURGESS$ svn status
M tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml
M tika-parsers/pom.xml
? tika-parsers/src/main/java/org/apache/tika/parser/mat
? tika-parsers/src/test/java/org/apache/tika/parser/mat
? tika-parsers/src/test/resources/test-documents/breidamerkurjokull_radar_profiles_2009.mat
- Ann Burgess
On June 6, 2014, 11:07 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 6, 2014, 11:07 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> TIKA-1327.aburgess.140606.patch.txt
> https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 6, 2014, 11:07 p.m.)
Review request for tika and Chris Mattmann.
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs
-----
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments (updated)
----------------
TIKA-1327.aburgess.140606.patch.txt
https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 6, 2014, 11:06 p.m.)
Review request for tika and Chris Mattmann.
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs
-----
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments (updated)
----------------
Parser File
https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
Unit Test
https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
TIKA-1327.aburgess.140606.patch.txt
https://reviews.apache.org/media/uploaded/files/2014/06/06/3babeb42-6e15-4d31-ae7d-9dc7ef4c5f65__TIKA-1327.aburgess.140606.patch.txt
Thanks,
Ann Burgess
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Matthias Krueger <co...@mkr.io>.
> On June 4, 2014, 11:25 p.m., Matthias Krueger wrote:
> > The Matlab MIME types used seem to be application/x-matlab-data or application/matlab-mat.
> >
> > Would it make sense to add them to the mime XML for detection?
> >
> > <mime-type type="application/x-matlab-data">
> > <comment>MATLAB data file</comment>
> > <alias type="application/matlab-mat"/>
> > <magic priority="50">
> > <match value="MATLAB" type="string" offset="0"/>
> > </magic>
> > <glob pattern="*.mat"/>
> > </mime-type>
> >
> >
>
> Chris Mattmann wrote:
> +1 this makes a ton of sense to add IMO.
>
> Nick Burch wrote:
> There's some odd whitespace going on - we normally use 4 spaces and no tabs.
>
> When outputting the variables, it would probably make sense to put each one into either a paragraph or a list, so that we get helpful output in html mode as well as text mode
>
> With that in place, it would then be possible to have a unit test that checked the html output, as well as the current text one
>
> Also on testing, I think at least some of the tests have an implementation of assertContains, which generally gives a more helpful failure message than assertTrue(s.contains(...)) does, might be worth looking into that?
>
> Ann Burgess wrote:
> Great input - thank you! I will integrate both and upload the diff.
This is on a good way, some quick additional comments:
* I tested with the files in https://github.com/scipy/scipy/tree/master/scipy/io/matlab/tests/data. JMatIO only support MATLAB 5 files. This could be added as a quick comment or javadoc.
* I think Tika is based on JDK 1.6. I don't see a reason for the test to take care and always just return-succeeding on JDK 1.5.
- Matthias
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44773
-----------------------------------------------------------
On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 10:23 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> Parser File
> https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
> Unit Test
> https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Chris Mattmann <ma...@apache.org>.
> On June 4, 2014, 11:25 p.m., Matthias Krueger wrote:
> > The Matlab MIME types used seem to be application/x-matlab-data or application/matlab-mat.
> >
> > Would it make sense to add them to the mime XML for detection?
> >
> > <mime-type type="application/x-matlab-data">
> > <comment>MATLAB data file</comment>
> > <alias type="application/matlab-mat"/>
> > <magic priority="50">
> > <match value="MATLAB" type="string" offset="0"/>
> > </magic>
> > <glob pattern="*.mat"/>
> > </mime-type>
> >
> >
+1 this makes a ton of sense to add IMO.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44773
-----------------------------------------------------------
On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 10:23 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> Parser File
> https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
> Unit Test
> https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Nick Burch <ni...@apache.org>.
> On June 4, 2014, 11:25 p.m., Matthias Krueger wrote:
> > The Matlab MIME types used seem to be application/x-matlab-data or application/matlab-mat.
> >
> > Would it make sense to add them to the mime XML for detection?
> >
> > <mime-type type="application/x-matlab-data">
> > <comment>MATLAB data file</comment>
> > <alias type="application/matlab-mat"/>
> > <magic priority="50">
> > <match value="MATLAB" type="string" offset="0"/>
> > </magic>
> > <glob pattern="*.mat"/>
> > </mime-type>
> >
> >
>
> Chris Mattmann wrote:
> +1 this makes a ton of sense to add IMO.
There's some odd whitespace going on - we normally use 4 spaces and no tabs.
When outputting the variables, it would probably make sense to put each one into either a paragraph or a list, so that we get helpful output in html mode as well as text mode
With that in place, it would then be possible to have a unit test that checked the html output, as well as the current text one
Also on testing, I think at least some of the tests have an implementation of assertContains, which generally gives a more helpful failure message than assertTrue(s.contains(...)) does, might be worth looking into that?
- Nick
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44773
-----------------------------------------------------------
On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 10:23 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> Parser File
> https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
> Unit Test
> https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Hi Annie,
Can you please create a JIRA issue for this, and also please create
a diff against the Tika trunk by doing the following:
0. create JIRA issue for Matlab parser
1. svn co http://svn.apache.org/repos/asf/tika/trunk tika
2. cd tika
3. drop your Matlab parser files in e.g.,
tika-parsers/src/main/java/org/apache/tika/parser/matlab
4. update file packages, etc.
5. svn status (files look ok?)
6. svn diff > TIKA-xxx.aburgess.yyMMdd.patch.txt (where xxx is the JIRA
issue id from 0.)
Then if you attach the diff to ReviewBoard I can annotate the lines etc
with comments. THanks! Also once you create the JIRA issue I will help
get it into the sources.
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: Ann Burgess <an...@gmail.com>
Reply-To: "dev@tika.apache.org" <de...@tika.apache.org>, "Bryant, Ann C
(398J-Affiliate)" <an...@gmail.com>
Date: Thursday, June 5, 2014 11:37 AM
To: Chris Mattmann <ma...@apache.org>
Cc: Matthias Krueger <co...@mkr.io>, tika <de...@tika.apache.org>, "Bryant,
Ann C (398J-Affiliate)" <an...@gmail.com>, Nick Burch
<ni...@apache.org>
Subject: Re: Review Request 22246: New parser for Matlab .mat files
>
>
>> On June 4, 2014, 11:25 p.m., Matthias Krueger wrote:
>> > The Matlab MIME types used seem to be application/x-matlab-data or
>>application/matlab-mat.
>> >
>> > Would it make sense to add them to the mime XML for detection?
>> >
>> > <mime-type type="application/x-matlab-data">
>> > <comment>MATLAB data file</comment>
>> > <alias type="application/matlab-mat"/>
>> > <magic priority="50">
>> > <match value="MATLAB" type="string" offset="0"/>
>> > </magic>
>> > <glob pattern="*.mat"/>
>> > </mime-type>
>> >
>> >
>>
>> Chris Mattmann wrote:
>> +1 this makes a ton of sense to add IMO.
>>
>> Nick Burch wrote:
>> There's some odd whitespace going on - we normally use 4 spaces and
>>no tabs.
>>
>> When outputting the variables, it would probably make sense to put
>>each one into either a paragraph or a list, so that we get helpful
>>output in html mode as well as text mode
>>
>> With that in place, it would then be possible to have a unit test
>>that checked the html output, as well as the current text one
>>
>> Also on testing, I think at least some of the tests have an
>>implementation of assertContains, which generally gives a more helpful
>>failure message than assertTrue(s.contains(...)) does, might be worth
>>looking into that?
>
>Great input - thank you! I will integrate both and upload the diff.
>
>
>- Ann
>
>
>-----------------------------------------------------------
>This is an automatically generated e-mail. To reply, visit:
>https://reviews.apache.org/r/22246/#review44773
>-----------------------------------------------------------
>
>
>On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>>
>> -----------------------------------------------------------
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/22246/
>> -----------------------------------------------------------
>>
>> (Updated June 4, 2014, 10:23 p.m.)
>>
>>
>> Review request for tika and Chris Mattmann.
>>
>>
>> Repository: tika
>>
>>
>> Description
>> -------
>>
>> This is a new parser for Matlab .mat files. The parser utilizes the
>>JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through
>>Maven Central. The text output from this parser provides variable names
>>and dimensions that are both inside and outside of data structures, but
>>does NOT provide the actual data values within each .mat file.
>>
>>
>> Diffs
>> -----
>>
>>
>> Diff: https://reviews.apache.org/r/22246/diff/
>>
>>
>> Testing
>> -------
>>
>> Successfully run a basic unit test that checks both --text and
>>--metadata parser output.
>>
>>
>> File Attachments
>> ----------------
>>
>> Parser File
>>
>>https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-
>>4fbc-b348-6a4db8907f6b__MatParser.java
>> Unit Test
>>
>>https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-
>>4830-b441-532c28c3c78e__MatParserTest.java
>>
>>
>> Thanks,
>>
>> Ann Burgess
>>
>>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
> On June 4, 2014, 11:25 p.m., Matthias Krueger wrote:
> > The Matlab MIME types used seem to be application/x-matlab-data or application/matlab-mat.
> >
> > Would it make sense to add them to the mime XML for detection?
> >
> > <mime-type type="application/x-matlab-data">
> > <comment>MATLAB data file</comment>
> > <alias type="application/matlab-mat"/>
> > <magic priority="50">
> > <match value="MATLAB" type="string" offset="0"/>
> > </magic>
> > <glob pattern="*.mat"/>
> > </mime-type>
> >
> >
>
> Chris Mattmann wrote:
> +1 this makes a ton of sense to add IMO.
>
> Nick Burch wrote:
> There's some odd whitespace going on - we normally use 4 spaces and no tabs.
>
> When outputting the variables, it would probably make sense to put each one into either a paragraph or a list, so that we get helpful output in html mode as well as text mode
>
> With that in place, it would then be possible to have a unit test that checked the html output, as well as the current text one
>
> Also on testing, I think at least some of the tests have an implementation of assertContains, which generally gives a more helpful failure message than assertTrue(s.contains(...)) does, might be worth looking into that?
Great input - thank you! I will integrate both and upload the diff.
- Ann
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44773
-----------------------------------------------------------
On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 10:23 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> Parser File
> https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
> Unit Test
> https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
> On June 4, 2014, 11:25 p.m., Matthias Krueger wrote:
> > The Matlab MIME types used seem to be application/x-matlab-data or application/matlab-mat.
> >
> > Would it make sense to add them to the mime XML for detection?
> >
> > <mime-type type="application/x-matlab-data">
> > <comment>MATLAB data file</comment>
> > <alias type="application/matlab-mat"/>
> > <magic priority="50">
> > <match value="MATLAB" type="string" offset="0"/>
> > </magic>
> > <glob pattern="*.mat"/>
> > </mime-type>
> >
> >
>
> Chris Mattmann wrote:
> +1 this makes a ton of sense to add IMO.
>
> Nick Burch wrote:
> There's some odd whitespace going on - we normally use 4 spaces and no tabs.
>
> When outputting the variables, it would probably make sense to put each one into either a paragraph or a list, so that we get helpful output in html mode as well as text mode
>
> With that in place, it would then be possible to have a unit test that checked the html output, as well as the current text one
>
> Also on testing, I think at least some of the tests have an implementation of assertContains, which generally gives a more helpful failure message than assertTrue(s.contains(...)) does, might be worth looking into that?
>
> Ann Burgess wrote:
> Great input - thank you! I will integrate both and upload the diff.
>
> Matthias Krueger wrote:
> This is on a good way, some quick additional comments:
> * I tested with the files in https://github.com/scipy/scipy/tree/master/scipy/io/matlab/tests/data. JMatIO only support MATLAB 5 files. This could be added as a quick comment or javadoc.
> * I think Tika is based on JDK 1.6. I don't see a reason for the test to take care and always just return-succeeding on JDK 1.5.
+1 Matthias.
- Ann
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44773
-----------------------------------------------------------
On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 10:23 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> Parser File
> https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
> Unit Test
> https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Matthias Krueger <co...@mkr.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/#review44773
-----------------------------------------------------------
The Matlab MIME types used seem to be application/x-matlab-data or application/matlab-mat.
Would it make sense to add them to the mime XML for detection?
<mime-type type="application/x-matlab-data">
<comment>MATLAB data file</comment>
<alias type="application/matlab-mat"/>
<magic priority="50">
<match value="MATLAB" type="string" offset="0"/>
</magic>
<glob pattern="*.mat"/>
</mime-type>
- Matthias Krueger
On June 4, 2014, 10:23 p.m., Ann Burgess wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22246/
> -----------------------------------------------------------
>
> (Updated June 4, 2014, 10:23 p.m.)
>
>
> Review request for tika and Chris Mattmann.
>
>
> Repository: tika
>
>
> Description
> -------
>
> This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/22246/diff/
>
>
> Testing
> -------
>
> Successfully run a basic unit test that checks both --text and --metadata parser output.
>
>
> File Attachments
> ----------------
>
> Parser File
> https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
> Unit Test
> https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
>
>
> Thanks,
>
> Ann Burgess
>
>
Re: Review Request 22246: New parser for Matlab .mat files
Posted by Ann Burgess <an...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22246/
-----------------------------------------------------------
(Updated June 4, 2014, 10:23 p.m.)
Review request for tika and Chris Mattmann.
Repository: tika
Description
-------
This is a new parser for Matlab .mat files. The parser utilizes the JmatIO, Matlab's MAT-file I/O API in JAVA. JmatIO is available through Maven Central. The text output from this parser provides variable names and dimensions that are both inside and outside of data structures, but does NOT provide the actual data values within each .mat file.
Diffs
-----
Diff: https://reviews.apache.org/r/22246/diff/
Testing
-------
Successfully run a basic unit test that checks both --text and --metadata parser output.
File Attachments (updated)
----------------
Parser File
https://reviews.apache.org/media/uploaded/files/2014/06/04/cb39636d-ec53-4fbc-b348-6a4db8907f6b__MatParser.java
Unit Test
https://reviews.apache.org/media/uploaded/files/2014/06/04/bbff8c6b-caa1-4830-b441-532c28c3c78e__MatParserTest.java
Thanks,
Ann Burgess