You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by Apache Wiki <wi...@apache.org> on 2015/03/25 23:14:43 UTC

[Tika Wiki] Trivial Update of "ScientificDataFormatTextRepresentation" by AnnieBurgess

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Tika Wiki" for change notification.

The "ScientificDataFormatTextRepresentation" page has been changed by AnnieBurgess:
https://wiki.apache.org/tika/ScientificDataFormatTextRepresentation

New page:
= Scientific Data Text Representation =

Tika now parses scientific data files such as HDF, NetCDF, and MatLab. Currently, the parsed output for these files is metadata.  While there is a 'text/ -t' output for NetCDF files, it includes variable names and dimensions, which is still in its essence, metadata. Enabling the ''data'' themselves to be parsed would enable a new paradigm in search.  

'''NetCDF Example''' 

A netCDF classic or 64-bit offset dataset is stored as a single file comprising two parts:
 * a header, containing all the information about dimensions, attributes, and variables except for the variable data;
 * a data part, comprising fixed-size data, containing the data for variables that don't have an unlimited dimension; and variable-size data, containing the data for variables that have an unlimited dimension.

The NetCDFparser currently extracts the "header part". 
 * -text extracts file Dimensions and Variables
 * -metadata extracts Global Attributes

We want the option to extract the "data part" of NetCDF files. 

'''JIRA Issue: TIKA-1577'''

Lets use the NetCDF test file for our dev testing: tika/tika-parsers/src/test/resources/test-documents/sresa1b_ncar_ccsm3_0_run1_200001.nc


Suggestions include: 

Use nested tables in HTML5:

http://stackoverflow.com/questions/10297874/html5-validity-of-nested-tables