You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@tika.apache.org by Ilya Zavorin <iz...@caci.com> on 2011/03/08 22:38:00 UTC

can Tika handle filenames with Unicode characters?

I was trying to run the following command:

java -jar "C:\Code\ATEK\CT\apache-tika-0.9\tika-app-0.9.jar" --xml "C:\Data\ATEK\CT\Tests\Example Documents\Urdu امریکہ کےانتباہ کےبعدبھارت میں سکیورٹی میں اضافہ.doc" > "C:\Data\ATEK\CT\Tests\Example Documents_OUT_XML\Urdu امریکہ کےانتباہ کےبعدبھارت میں سکیورٹی میں اضافہ.doc.OUT.xml"

this command was specified in a batch file that was saved as an UTF-8 file without the BOM.

This produced an empty output file with the following name:

Urdu ╪º┘à╪▒█î┌⌐█ü ┌⌐█Æ╪º┘å╪¬╪¿╪º█ü ┌⌐█Æ╪¿╪╣╪»╪¿┌╛╪º╪▒╪¬ ┘à█î┌║ ╪│┌⌐█î┘ê╪▒┘╣█î ┘à█î┌║ ╪º╪╢╪º┘ü█ü.doc.OUT.xml

It also generated the following exception:

Exception in thread "main" java.net.MalformedURLException: unknown protocol: c
        at java.net.URL.<init>(URL.java:590)
        at java.net.URL.<init>(URL.java:480)
        at java.net.URL.<init>(URL.java:429)
        at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:298)
        at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:91)

Is this caused by the Unicode characters in either the input or the output filepath? Is there a way of processing such files without renaming them?

Thanks,

Mr. Ilya Zavorin, Ph.D.
Principal Research Analyst
Knowledge and Information Management Division
CACI International
4831 Walden Lane
Lanham, MD 20706
ph: 1-301-306-2859
fx: 1-301-306-8201
izavorin@caci.com
www.caci.com

Re: TikaApp "-s" option

Posted by Dave Meikle <lo...@gmail.com>.

Hi Zabrane,

On Thursday, 17 March 2011, Zabrane Mickael <za...@gmail.com> wrote:
> Could someone please explain me the "-s | --server" option and how
> to use it?
>

This option is part of the Tika Network Server functionality being
implemented as part of TIKA-593[1].  At present the -server command
does not do anything in the code, with the -p or -port activating a
socket server at the moment.

The feature is currently being revisited with a JAX-RS server, so
check out the ticket for more details if this interests you.

Regards,
Dave

[1] https://issues.apache.org/Jura/browse/TIKA-593

TikaApp "-s" option

Posted by Zabrane Mickael <za...@gmail.com>.

Hi guys,

Could someone please explain me the "-s | --server" option and how 
to use it ?

$ java -jar tika-app.jar -s
Exception in thread "main" java.net.MalformedURLException: no protocol: -s
	at java.net.URL.<init>(URL.java:567)
	at java.net.URL.<init>(URL.java:464)
	at java.net.URL.<init>(URL.java:413)
	at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:298)
	at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:91)
 
Regards
Zab

Re: can Tika handle filenames with Unicode characters?

Posted by Jukka Zitting <jz...@adobe.com>.

Hi,

On 03/08/2011 10:38 PM, Ilya Zavorin wrote:
> It also generated the following exception:
>
> Exception in thread "main" java.net.MalformedURLException: unknown
> protocol: c

This is a result of Tika not finding the file you specified, and then 
trying to resolve the given path as a URL. We probably should detect 
"c:\" and similar prefixes as special cases that should never be 
interpreted as a URL.

As mentioned by Benson, this problem most likely has nothing to do with 
Tika as such, it's just the way the command line interprets the entered 
characters.

To verify this, try replacing the 'java -jar 
"C:\Code\ATEK\CT\apache-tika-0.9\tika-app-0.9.jar" --xml' part of your 
command line with a simpler command like just 'dir'. I predict that 
you'll get a "File Not Found" error message as a result.

-- 
Jukka Zitting

Re: can Tika handle filenames with Unicode characters?

Posted by Benson Margulies <bi...@gmail.com>.

Ilya,

Until you stop using the Tika command line and actually use the TIka
API, you are at the mercy of Windows pranks. Java famously did some
really dumb things to support WIndows 3.1 that may never have  been
undone.


On Tue, Mar 8, 2011 at 5:39 PM, Ilya Zavorin <iz...@caci.com> wrote:
> Well, I tried invoking the same command from a .NET application using Process() and got the same result.
>
>
> Mr. Ilya Zavorin, Ph.D.
> Principal Research Analyst
> Knowledge and Information Management Division
> CACI International
> 4831 Walden Lane
> Lanham, MD 20706
> ph: 1-301-306-2859
> fx: 1-301-306-8201
> izavorin@caci.com
> www.caci.com
>
>
> -----Original Message-----
> From: Benson Margulies [mailto:bimargulies@gmail.com]
> Sent: Tuesday, March 08, 2011 4:54 PM
> To: user@tika.apache.org
> Subject: Re: can Tika handle filenames with Unicode characters?
>
> Ilya,
>
> Since you are at the CMD box prompt, you are at the mercy of the the
> fact that the CMD box only works in the 'ascii' mode (current Ansi
> Code Page), not in Unicode. I recommend that you try this out with a
> littte java program that calls the Tika API, and I expect, though
> guarantee, that it will work.
>
>
>
> On Tue, Mar 8, 2011 at 4:38 PM, Ilya Zavorin <iz...@caci.com> wrote:
>> I was trying to run the following command:
>>
>>
>>
>> java -jar "C:\Code\ATEK\CT\apache-tika-0.9\tika-app-0.9.jar" --xml
>> "C:\Data\ATEK\CT\Tests\Example Documents\Urdu امریکہ کےانتباہ کےبعدبھارت میں
>> سکیورٹی میں اضافہ.doc" > "C:\Data\ATEK\CT\Tests\Example
>> Documents_OUT_XML\Urdu امریکہ کےانتباہ کےبعدبھارت میں سکیورٹی میں
>> اضافہ.doc.OUT.xml"
>>
>>
>>
>> this command was specified in a batch file that was saved as an UTF-8 file
>> without the BOM.
>>
>>
>>
>> This produced an empty output file with the following name:
>>
>>
>>
>> Urdu ╪º┘à╪▒█î┌⌐█ü ┌⌐█Æ╪º┘å╪¬╪¿╪º█ü ┌⌐█Æ╪¿╪╣╪»╪¿┌╛╪º╪▒╪¬ ┘à█î┌║
>> ╪│┌⌐█î┘ê╪▒┘╣█î ┘à█î┌║ ╪º╪╢╪º┘ü█ü.doc.OUT.xml
>>
>> It also generated the following exception:
>>
>>
>>
>> Exception in thread "main" java.net.MalformedURLException: unknown protocol:
>> c
>>
>>         at java.net.URL.<init>(URL.java:590)
>>
>>         at java.net.URL.<init>(URL.java:480)
>>
>>         at java.net.URL.<init>(URL.java:429)
>>
>>         at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:298)
>>
>>         at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:91)
>>
>>
>>
>> Is this caused by the Unicode characters in either the input or the output
>> filepath? Is there a way of processing such files without renaming them?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Mr. Ilya Zavorin, Ph.D.
>>
>> Principal Research Analyst
>>
>> Knowledge and Information Management Division
>>
>> CACI International
>>
>> 4831 Walden Lane
>>
>> Lanham, MD 20706
>>
>> ph: 1-301-306-2859
>>
>> fx: 1-301-306-8201
>>
>> izavorin@caci.com
>>
>> www.caci.com
>>
>>
>

RE: can Tika handle filenames with Unicode characters?

Posted by Ilya Zavorin <iz...@caci.com>.

Well, I tried invoking the same command from a .NET application using Process() and got the same result.


Mr. Ilya Zavorin, Ph.D.
Principal Research Analyst
Knowledge and Information Management Division
CACI International
4831 Walden Lane
Lanham, MD 20706
ph: 1-301-306-2859
fx: 1-301-306-8201
izavorin@caci.com
www.caci.com


-----Original Message-----
From: Benson Margulies [mailto:bimargulies@gmail.com] 
Sent: Tuesday, March 08, 2011 4:54 PM
To: user@tika.apache.org
Subject: Re: can Tika handle filenames with Unicode characters?

Ilya,

Since you are at the CMD box prompt, you are at the mercy of the the
fact that the CMD box only works in the 'ascii' mode (current Ansi
Code Page), not in Unicode. I recommend that you try this out with a
littte java program that calls the Tika API, and I expect, though
guarantee, that it will work.



On Tue, Mar 8, 2011 at 4:38 PM, Ilya Zavorin <iz...@caci.com> wrote:
> I was trying to run the following command:
>
>
>
> java -jar "C:\Code\ATEK\CT\apache-tika-0.9\tika-app-0.9.jar" --xml
> "C:\Data\ATEK\CT\Tests\Example Documents\Urdu امریکہ کےانتباہ کےبعدبھارت میں
> سکیورٹی میں اضافہ.doc" > "C:\Data\ATEK\CT\Tests\Example
> Documents_OUT_XML\Urdu امریکہ کےانتباہ کےبعدبھارت میں سکیورٹی میں
> اضافہ.doc.OUT.xml"
>
>
>
> this command was specified in a batch file that was saved as an UTF-8 file
> without the BOM.
>
>
>
> This produced an empty output file with the following name:
>
>
>
> Urdu ╪º┘à╪▒█î┌⌐█ü ┌⌐█Æ╪º┘å╪¬╪¿╪º█ü ┌⌐█Æ╪¿╪╣╪»╪¿┌╛╪º╪▒╪¬ ┘à█î┌║
> ╪│┌⌐█î┘ê╪▒┘╣█î ┘à█î┌║ ╪º╪╢╪º┘ü█ü.doc.OUT.xml
>
> It also generated the following exception:
>
>
>
> Exception in thread "main" java.net.MalformedURLException: unknown protocol:
> c
>
>         at java.net.URL.<init>(URL.java:590)
>
>         at java.net.URL.<init>(URL.java:480)
>
>         at java.net.URL.<init>(URL.java:429)
>
>         at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:298)
>
>         at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:91)
>
>
>
> Is this caused by the Unicode characters in either the input or the output
> filepath? Is there a way of processing such files without renaming them?
>
>
>
> Thanks,
>
>
>
> Mr. Ilya Zavorin, Ph.D.
>
> Principal Research Analyst
>
> Knowledge and Information Management Division
>
> CACI International
>
> 4831 Walden Lane
>
> Lanham, MD 20706
>
> ph: 1-301-306-2859
>
> fx: 1-301-306-8201
>
> izavorin@caci.com
>
> www.caci.com
>
>

Re: can Tika handle filenames with Unicode characters?

Posted by Benson Margulies <bi...@gmail.com>.

Ilya,

Since you are at the CMD box prompt, you are at the mercy of the the
fact that the CMD box only works in the 'ascii' mode (current Ansi
Code Page), not in Unicode. I recommend that you try this out with a
littte java program that calls the Tika API, and I expect, though
guarantee, that it will work.



On Tue, Mar 8, 2011 at 4:38 PM, Ilya Zavorin <iz...@caci.com> wrote:
> I was trying to run the following command:
>
>
>
> java -jar "C:\Code\ATEK\CT\apache-tika-0.9\tika-app-0.9.jar" --xml
> "C:\Data\ATEK\CT\Tests\Example Documents\Urdu امریکہ کےانتباہ کےبعدبھارت میں
> سکیورٹی میں اضافہ.doc" > "C:\Data\ATEK\CT\Tests\Example
> Documents_OUT_XML\Urdu امریکہ کےانتباہ کےبعدبھارت میں سکیورٹی میں
> اضافہ.doc.OUT.xml"
>
>
>
> this command was specified in a batch file that was saved as an UTF-8 file
> without the BOM.
>
>
>
> This produced an empty output file with the following name:
>
>
>
> Urdu ╪º┘à╪▒█î┌⌐█ü ┌⌐█Æ╪º┘å╪¬╪¿╪º█ü ┌⌐█Æ╪¿╪╣╪»╪¿┌╛╪º╪▒╪¬ ┘à█î┌║
> ╪│┌⌐█î┘ê╪▒┘╣█î ┘à█î┌║ ╪º╪╢╪º┘ü█ü.doc.OUT.xml
>
> It also generated the following exception:
>
>
>
> Exception in thread "main" java.net.MalformedURLException: unknown protocol:
> c
>
>         at java.net.URL.<init>(URL.java:590)
>
>         at java.net.URL.<init>(URL.java:480)
>
>         at java.net.URL.<init>(URL.java:429)
>
>         at org.apache.tika.cli.TikaCLI.process(TikaCLI.java:298)
>
>         at org.apache.tika.cli.TikaCLI.main(TikaCLI.java:91)
>
>
>
> Is this caused by the Unicode characters in either the input or the output
> filepath? Is there a way of processing such files without renaming them?
>
>
>
> Thanks,
>
>
>
> Mr. Ilya Zavorin, Ph.D.
>
> Principal Research Analyst
>
> Knowledge and Information Management Division
>
> CACI International
>
> 4831 Walden Lane
>
> Lanham, MD 20706
>
> ph: 1-301-306-2859
>
> fx: 1-301-306-8201
>
> izavorin@caci.com
>
> www.caci.com
>
>