You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by Apache Wiki <wi...@apache.org> on 2015/03/25 23:14:43 UTC
[Tika Wiki] Trivial Update of "ScientificDataFormatTextRepresentation" by AnnieBurgess
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Tika Wiki" for change notification.
The "ScientificDataFormatTextRepresentation" page has been changed by AnnieBurgess:
https://wiki.apache.org/tika/ScientificDataFormatTextRepresentation
New page:
= Scientific Data Text Representation =
Tika now parses scientific data files such as HDF, NetCDF, and MatLab. Currently, the parsed output for these files is metadata. While there is a 'text/ -t' output for NetCDF files, it includes variable names and dimensions, which is still in its essence, metadata. Enabling the ''data'' themselves to be parsed would enable a new paradigm in search.
'''NetCDF Example'''
A netCDF classic or 64-bit offset dataset is stored as a single file comprising two parts:
* a header, containing all the information about dimensions, attributes, and variables except for the variable data;
* a data part, comprising fixed-size data, containing the data for variables that don't have an unlimited dimension; and variable-size data, containing the data for variables that have an unlimited dimension.
The NetCDFparser currently extracts the "header part".
* -text extracts file Dimensions and Variables
* -metadata extracts Global Attributes
We want the option to extract the "data part" of NetCDF files.
'''JIRA Issue: TIKA-1577'''
Lets use the NetCDF test file for our dev testing: tika/tika-parsers/src/test/resources/test-documents/sresa1b_ncar_ccsm3_0_run1_200001.nc
Suggestions include:
Use nested tables in HTML5:
http://stackoverflow.com/questions/10297874/html5-validity-of-nested-tables