Solr custom file types

[Mike Domzalski <mi...@oracle.com>]

Hi all

Basically, I am a Solr newbie and have had 0 experience with this as our Solr expert left the company. We are receiving a file from a client that is a proprietary file (.gnet). I don't have access to the application in which it was generated from.

When uploading to Solr we receive the following error

SOLR Log
solr-cloud.log: {"msg":"2022-01-19 08:10:06.915 ERROR (qtp349420578-3516) [c:<collection> s:shard2 r:core_node5 x:<redacted>] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: java.lang.NoClassDefFoundError: ucar/nc2/NetcdfFile"}

Our App logging
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr/<collection>: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /solr/<collection>/update/extract. Reason:
<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NoClassDefFoundError: ucar/nc2/NetcdfFile
        at org.apache.tika.parser.hdf.HDFParser.parse(HDFParser.java:88)
        at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)

Other normal file types works (e.g. doc, pdf, zip)

I cannot open or edit the file to see what fields are in there to index so is there a way to be ignore this file and not index?
If not, is there anything else I can do to handle this file type or ignore this error

TIA
--
Thanks, Mike

Re: Solr custom file types

[Mike Drob <md...@mdrob.com>]

I suspect the missing libraries are somehow related to
https://github.com/Unidata/netcdf-java

Mike

On Wed, Jan 26, 2022 at 6:32 PM Shawn Heisey <ap...@elyograg.org> wrote:

> On 1/26/2022 5:10 PM, Mike Domzalski wrote:
> > <pre>    Server Error</pre></p><h3>Caused
> by:</h3><pre>java.lang.NoClassDefFoundError: ucar/nc2/NetcdfFile
> >          at org.apache.tika.parser.hdf.HDFParser.parse(HDFParser.java:88)
> >          at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>
> The extracting request handler in Solr utilizes Tika to handle documents.
>
> Tika has indicated that a class named ucar/nc2/NetcdfFile is required to
> extract the file.  That class is not included in Solr and it has not
> been provided via other means.  Whatever the file format is, the parts
> of Tika that have been added to Solr are not capable of opening the
> file, more is needed.
>
> I have no idea whether Solr will know how to index the extracted data
> even if you provide the jar(s) needed so Tika can open the file.
>
> Thanks,
> Shawn
>

Re: Solr custom file types

[Shawn Heisey <ap...@elyograg.org>]

On 1/26/2022 5:10 PM, Mike Domzalski wrote:
> <pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NoClassDefFoundError: ucar/nc2/NetcdfFile
>          at org.apache.tika.parser.hdf.HDFParser.parse(HDFParser.java:88)
>          at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)

The extracting request handler in Solr utilizes Tika to handle documents.

Tika has indicated that a class named ucar/nc2/NetcdfFile is required to 
extract the file.  That class is not included in Solr and it has not 
been provided via other means.  Whatever the file format is, the parts 
of Tika that have been added to Solr are not capable of opening the 
file, more is needed.

I have no idea whether Solr will know how to index the extracted data 
even if you provide the jar(s) needed so Tika can open the file.

Thanks,
Shawn