You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tika.apache.org by Mark Kerzner <ma...@shmsoft.com> on 2016/09/28 05:22:09 UTC
Code parser?
Hi,
I want Tika to parse source code files, primarily Java. Is there anything
special I need to do?
Right now my code seems to recognize Java as Microsoft Office Document.
Thank you,
Mark
Mark Kerzner, SHMsoft <http://shmsoft.com/>,
Book a call with me here <http://www.meetme.so/markkerzner>
Mobile: 713-724-2534
Skype: mark.kerzner1
<http://shmsoft.com/>
Re: Code parser?
Posted by Mark Kerzner <ma...@shmsoft.com>.
Nick, Marcus,
thank you for your help. It works great, and one of the problems that I saw
was indeed with my code, not Tika.
Mark
Mark Kerzner, SHMsoft <http://shmsoft.com/>,
Book a call with me here <http://www.meetme.so/markkerzner>
Mobile: 713-724-2534
Skype: mark.kerzner1
<http://shmsoft.com/>
On Thu, Sep 29, 2016 at 5:21 PM, Nick Burch <ap...@gagravarr.org> wrote:
> On Wed, 28 Sep 2016, Mark Kerzner wrote:
>
>> probably yes, but how do I tell it which parser to use? Today, I just do
>> that
>>
>> String text = tika.parseToString(inputStream, metadata);
>>
>> and it know the parser.
>>
>
> That might be your issue. It's quite hard to identify the language of a
> piece of source code from just the first few hundred bytes of text. If you
> tell Tika the filename, including the extension, it'll have much more luck
> spotting the file is code and using the appropriate parser!
>
> (Binary files often have common magic at/near the start that helps Tika
> identify the file type, source code is text based and lacks that)
>
> Nick
>
Re: Code parser?
Posted by Nick Burch <ap...@gagravarr.org>.
On Wed, 28 Sep 2016, Mark Kerzner wrote:
> probably yes, but how do I tell it which parser to use? Today, I just do
> that
>
> String text = tika.parseToString(inputStream, metadata);
>
> and it know the parser.
That might be your issue. It's quite hard to identify the language of a
piece of source code from just the first few hundred bytes of text. If you
tell Tika the filename, including the extension, it'll have much more luck
spotting the file is code and using the appropriate parser!
(Binary files often have common magic at/near the start that helps Tika
identify the file type, source code is text based and lacks that)
Nick
Re: Code parser?
Posted by Mark Kerzner <ma...@shmsoft.com>.
Hi, Marcus,
probably yes, but how do I tell it which parser to use? Today, I just do
that
String text = tika.parseToString(inputStream, metadata);
and it know the parser.
Thank you
Mark Kerzner, SHMsoft <http://shmsoft.com/>,
Book a call with me here <http://www.meetme.so/markkerzner>
Mobile: 713-724-2534
Skype: mark.kerzner1
<http://shmsoft.com/>
On Wed, Sep 28, 2016 at 2:56 AM, Markus Jelsma <ma...@openindex.io>
wrote:
> Hello Mark,
>
> Would SourceCodeParser suit your needs?
> https://tika.apache.org/1.13/api/org/apache/tika/parser/
> code/SourceCodeParser.html
>
> Regards,
> Markus
>
> -----Original message-----
> > From:Mark Kerzner <ma...@shmsoft.com>
> > Sent: Wednesday 28th September 2016 7:22
> > To: Tika User <us...@tika.apache.org>
> > Subject: Code parser?
> >
> > Hi,
> >
> > I want Tika to parse source code files, primarily Java. Is there
> anything special I need to do?
> >
> > Right now my code seems to recognize Java as Microsoft Office Document.
> >
> > Thank you,
> > Mark
> >
> > <br clear="all" />Mark Kerzner, SHMsoft <http://shmsoft.com/>,
> > Book a call with me here <http://www.meetme.so/markkerzner>
> > Mobile: 713-724-2534
> > Skype: mark.kerzner1
> > <http://shmsoft.com/>
>
RE: Code parser?
Posted by Markus Jelsma <ma...@openindex.io>.
Hello Mark,
Would SourceCodeParser suit your needs?
https://tika.apache.org/1.13/api/org/apache/tika/parser/code/SourceCodeParser.html
Regards,
Markus
-----Original message-----
> From:Mark Kerzner <ma...@shmsoft.com>
> Sent: Wednesday 28th September 2016 7:22
> To: Tika User <us...@tika.apache.org>
> Subject: Code parser?
>
> Hi,
>
> I want Tika to parse source code files, primarily Java. Is there anything special I need to do?
>
> Right now my code seems to recognize Java as Microsoft Office Document.
>
> Thank you,
> Mark
>
> <br clear="all" />Mark Kerzner, SHMsoft <http://shmsoft.com/>,
> Book a call with me here <http://www.meetme.so/markkerzner>
> Mobile: 713-724-2534
> Skype: mark.kerzner1
> <http://shmsoft.com/>