You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tika.apache.org by "Elif T. Kus" <el...@gmail.com> on 2010/04/12 15:01:43 UTC

Problem with encoding during installation on mac os x

Hello,

I downloaded tika-0.7.

During the installation two tests failed.

The errors show that there's a problem with the lower case i. The jvm was
set to use user.language tr (for Turkish). I suspected that was the case,
because Turkish is different than english in that we have two i's. One with
the dot and one without the dot. So the lower case "i with a dot"(i) has a
upper case "i with a dot" (İ). And the lower case "i without a dot" (ı) has
a upper case "i without a dot" (I). Which would be i-I in English.

So the failing tests made me suspect that the problem was that I use mac os
x with user.language set to tr (for Turkish). But setting the relevant props
to en in MAVEN_OPTS didn't have any effect.

export MAVEN_OPTS="-Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF8
-Duser.language=en -Duser.country=EN"

What can I do?

Failed tests:
  testDetect(org.apache.tika.detect.TypeDetectorTest)
  testLowerCase(org.apache.tika.mime.MediaTypeTest)

-------------------------------------------------------------------------------
Test set: org.apache.tika.detect.TypeDetectorTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.028 sec
<<< FAILURE!
testDetect(org.apache.tika.detect.TypeDetectorTest)  Time elapsed: 0.024
sec  <<< FAILURE!
junit.framework.AssertionFailedError: expected:<text/plain> but
was:<text/pla�n>
    at junit.framework.Assert.fail(Assert.java:47)
    at junit.framework.Assert.failNotEquals(Assert.java:282)
    at junit.framework.Assert.assertEquals(Assert.java:64)
    at junit.framework.Assert.assertEquals(Assert.java:71)
    at
org.apache.tika.detect.TypeDetectorTest.assertDetect(TypeDetectorTest.java:69)
    at
org.apache.tika.detect.TypeDetectorTest.testDetect(TypeDetectorTest.java:46)

-------------------------------------------------------------------------------
Test set: org.apache.tika.mime.MediaTypeTest
-------------------------------------------------------------------------------
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
<<< FAILURE!
testLowerCase(org.apache.tika.mime.MediaTypeTest)  Time elapsed: 0.001 sec
<<< FAILURE!
junit.framework.ComparisonFailure: expected:<...i...> but was:<...�...>
    at junit.framework.Assert.assertEquals(Assert.java:81)
    at junit.framework.Assert.assertEquals(Assert.java:87)
    at
org.apache.tika.mime.MediaTypeTest.testLowerCase(MediaTypeTest.java:52)


Platform
Max os x 10.6.3

Java properties

java.runtime.name: Java(TM) SE Runtime Environment
sun.boot.library.path:
/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries
java.vm.version: 14.3-b01-101
awt.nativeDoubleBuffering: true
gopherProxySet: false
mrj.build: 10M3025
java.vm.vendor: Apple Inc.
java.vendor.url: http://www.apple.com/
path.separator: :
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
file.encoding.pkg: sun.io
sun.java.launcher: SUN_STANDARD
user.country: TR
sun.os.patch.level: unknown
java.vm.specification.name: Java Virtual Machine Specification
user.dir: /Users/elif/workspace/JParser
java.runtime.version: 1.6.0_17-b04-248-10M3025
java.awt.graphicsenv: apple.awt.CGraphicsEnvironment
java.endorsed.dirs:
/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/endorsed
os.arch: x86_64
java.io.tmpdir: /var/folders/jU/jUxoh2kOHfKUjXUZkhwDBU+++TI/-Tmp-/
line.separator:

java.vm.specification.vendor: Sun Microsystems Inc.
os.name: Mac OS X
sun.jnu.encoding: MacTurkish
java.library.path:
.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
java.specification.name: Java Platform API Specification
java.class.version: 50.0
sun.management.compiler: HotSpot 64-Bit Server Compiler
os.version: 10.6.3
http.nonProxyHosts: local|*.local|169.254/16|*.169.254/16
user.home: /Users/elif
user.timezone:
java.awt.printerjob: apple.awt.CPrinterJob
file.encoding: UTF-8
java.specification.version: 1.6
java.class.path: /Users/elif/workspace/J....
user.name: elif
java.vm.specification.version: 1.0
java.home: /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
sun.arch.data.model: 64
user.language: tr
java.specification.vendor: Sun Microsystems Inc.
awt.toolkit: apple.awt.CToolkit
java.vm.info: mixed mode
java.version: 1.6.0_17
java.ext.dirs:
/Library/Java/Extensions:/System/Library/Java/Extensions:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/ext
sun.boot.class.path:
/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jsfd.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/ui.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/sunrsasign.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/charsets.jar
java.vendor: Apple Inc.
file.separator: /
java.vendor.url.bug: http://bugreport.apple.com/
sun.io.unicode.encoding: UnicodeLittle
sun.cpu.endian: little
mrj.version: 1060.1.6.0_17-248
socksNonProxyHosts: local|*.local|169.254/16|*.169.254/16
ftp.nonProxyHosts: local|*.local|169.254/16|*.169.254/16
sun.cpu.isalist:


Best,
Elif

Re: Problem with encoding during installation on mac os x

Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
+1 Jukka, nice!

Cheers,
Chris



On 4/12/10 7:46 AM, "Jukka Zitting" <ju...@gmail.com> wrote:

Hi,

On Mon, Apr 12, 2010 at 4:26 PM, Elif T. Kus <el...@gmail.com> wrote:
> I received the same errors.

I looked at this in more detail and it turns out to be a bug in Tika.
The media type handling code uses String.toLowerCase() to normalize
type names, which unfortunately is locale-specific, see TIKA-404 [1].

I've committed a fix to the Tika trunk in revision 933255 [2], and you
can use the patch at [3] to fix the 0.7 version.

[1] https://issues.apache.org/jira/browse/TIKA-404
[2] http://svn.apache.org/viewvc?view=revision&revision=933255
[3] http://svn.apache.org/viewvc/lucene/tika/trunk/tika-core/src/main/java/org/apache/tika/mime/MediaType.java?r1=933255&r2=933254&pathrev=933255&view=patch

BR,

Jukka Zitting



++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Re: Problem with encoding during installation on mac os x

Posted by "Elif T. Kus" <el...@gmail.com>.
Hi again,

Just to let you know. I had further failing tests. And this time they were
from the HtmlParser class so I thought it might need a more complicated
solution rather than just using Locale.ENGLISH . (Because I want to be able
to parse documents with Turkish characters.)

So I found another solution. I changed the language setting for my operating
system. And everything compiled fine. I used the jar from the command line
and it parsed a doc with Turkish characters successfully.

Apparently MAVEN_OPTS doesn't function as pointed out in the maven
documentation.

Kindly,
Elif



On Mon, Apr 12, 2010 at 7:05 PM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> 2010/4/12 Elif T. Kus <el...@gmail.com>:
> > The MimeTypesTest still failed.
> >
> > So I added the following and it worked:
> >
> > On line 196 changed:
> >
> >         - type = patterns.matches(name.toLowerCase());
> >         + type = patterns.matches(name.toLowerCase(Locale.ENGLISH));
> >
> > And on line 402:
> >
> >             name = name.toLowerCase();
> >             name = name.toLowerCase(Locale.ENGLISH);
> >
> > Now that test runs, too.
>
> Thanks! I committed these changes in revision 933295.
>
> BR,
>
> Jukka Zitting
>



-- 
-------------------------------
www.islerburada.com
İş arama motoru

Re: Problem with encoding during installation on mac os x

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

2010/4/12 Elif T. Kus <el...@gmail.com>:
> The MimeTypesTest still failed.
>
> So I added the following and it worked:
>
> On line 196 changed:
>
>         - type = patterns.matches(name.toLowerCase());
>         + type = patterns.matches(name.toLowerCase(Locale.ENGLISH));
>
> And on line 402:
>
>             name = name.toLowerCase();
>             name = name.toLowerCase(Locale.ENGLISH);
>
> Now that test runs, too.

Thanks! I committed these changes in revision 933295.

BR,

Jukka Zitting

Re: Problem with encoding during installation on mac os x

Posted by "Elif T. Kus" <el...@gmail.com>.
Hello,

The MimeTypesTest still failed.

So I added the following and it worked:

On line 196 changed:

        - type = patterns.matches(name.toLowerCase());
        + type = patterns.matches(name.toLowerCase(Locale.ENGLISH));

And on line 402:

            name = name.toLowerCase();
            name = name.toLowerCase(Locale.ENGLISH);

Now that test runs, too.

Kindly,
Elif

On Mon, Apr 12, 2010 at 5:46 PM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> On Mon, Apr 12, 2010 at 4:26 PM, Elif T. Kus <el...@gmail.com> wrote:
> > I received the same errors.
>
> I looked at this in more detail and it turns out to be a bug in Tika.
> The media type handling code uses String.toLowerCase() to normalize
> type names, which unfortunately is locale-specific, see TIKA-404 [1].
>
> I've committed a fix to the Tika trunk in revision 933255 [2], and you
> can use the patch at [3] to fix the 0.7 version.
>
> [1] https://issues.apache.org/jira/browse/TIKA-404
> [2] http://svn.apache.org/viewvc?view=revision&revision=933255
> [3]
> http://svn.apache.org/viewvc/lucene/tika/trunk/tika-core/src/main/java/org/apache/tika/mime/MediaType.java?r1=933255&r2=933254&pathrev=933255&view=patch
>
> BR,
>
> Jukka Zitting
>



-- 
-------------------------------
www.islerburada.com
İş arama motoru

Re: Problem with encoding during installation on mac os x

Posted by "Elif T. Kus" <el...@gmail.com>.
Thank you very much. :)


On Mon, Apr 12, 2010 at 5:46 PM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> On Mon, Apr 12, 2010 at 4:26 PM, Elif T. Kus <el...@gmail.com> wrote:
> > I received the same errors.
>
> I looked at this in more detail and it turns out to be a bug in Tika.
> The media type handling code uses String.toLowerCase() to normalize
> type names, which unfortunately is locale-specific, see TIKA-404 [1].
>
> I've committed a fix to the Tika trunk in revision 933255 [2], and you
> can use the patch at [3] to fix the 0.7 version.
>
> [1] https://issues.apache.org/jira/browse/TIKA-404
> [2] http://svn.apache.org/viewvc?view=revision&revision=933255
> [3]
> http://svn.apache.org/viewvc/lucene/tika/trunk/tika-core/src/main/java/org/apache/tika/mime/MediaType.java?r1=933255&r2=933254&pathrev=933255&view=patch
>
> BR,
>
> Jukka Zitting
>



-- 
-------------------------------
www.islerburada.com
İş arama motoru

Re: Problem with encoding during installation on mac os x

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Mon, Apr 12, 2010 at 4:26 PM, Elif T. Kus <el...@gmail.com> wrote:
> I received the same errors.

I looked at this in more detail and it turns out to be a bug in Tika.
The media type handling code uses String.toLowerCase() to normalize
type names, which unfortunately is locale-specific, see TIKA-404 [1].

I've committed a fix to the Tika trunk in revision 933255 [2], and you
can use the patch at [3] to fix the 0.7 version.

[1] https://issues.apache.org/jira/browse/TIKA-404
[2] http://svn.apache.org/viewvc?view=revision&revision=933255
[3] http://svn.apache.org/viewvc/lucene/tika/trunk/tika-core/src/main/java/org/apache/tika/mime/MediaType.java?r1=933255&r2=933254&pathrev=933255&view=patch

BR,

Jukka Zitting

Re: Problem with encoding during installation on mac os x

Posted by "Elif T. Kus" <el...@gmail.com>.
I used mvn -X clean install and I saw the encoding = UTF-8 setting
And I received the same errors.

I also added  <fork>true</fork> option to the maven-compiler-plugin
configuration in the tika-parent POM.
I received the same errors.

Elif

Details: mvn -X clean install | grep encoding

Default locale: en_EN, platform encoding: UTF8
[DEBUG]   (f) encoding = UTF-8
[DEBUG] properties used {env.VERSIONER_PERL_PREFER_32_BIT=yes,
java.vendor=Apple Inc., maven.compile.source=1.5,
sun.java.launcher=SUN_STANDARD, sun.management.compiler=HotSpot 64-Bit
Server Compiler, os.name=Mac OS X,
sun.boot.class.path=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jsfd.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/ui.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/sunrsasign.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/charsets.jar,
env.TMPDIR=/var/folders/jU/jUxoh2kOHfKUjXUZkhwDBU+++TI/-Tmp-/,
env.PWD=/Users/elif/DevelopmentLibraries/tika/svn/tika-site,
env.GDK_USE_XFT=1, env.LANG=tr_TR.UTF-8, java.vm.specification.vendor=Sun
Microsystems Inc., java.runtime.version=1.6.0_17-b04-248-10M3025,
env.Apple_PubSub_Socket_Render=/tmp/launch-4tCNW2/Render,
project.build.sourceEncoding=UTF-8, env.DISPLAY=/tmp/launch-0t3tnt/org.x:0,
user.name=elif, env.JAVA_MAIN_CLASS_49968=org.codehaus.classworlds.Launcher,
env.USER=elif, env.SHELL=/bin/bash, env.__CF_USER_TEXT_ENCODING=0x1F5:35:24,
awt.nativeDoubleBuffering=true, env.XDG_DATA_DIRS=/sw/share,
env.PATH=/sw/bin:/sw/sbin:/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/X11R6/bin,
user.language=en,
sun.boot.library.path=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries,
classworlds.conf=/usr/share/maven/bin/m2.conf, java.version=1.6.0_17,
user.timezone=Europe/Istanbul, organization.logo=
http://www.apache.org/images/asf_logo_wide.gif, sun.arch.data.model=64,
http.nonProxyHosts=local|*.local|169.254/16|*.169.254/16,
java.endorsed.dirs=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/endorsed,
sun.cpu.isalist=, sun.jnu.encoding=UTF-8, file.encoding.pkg=sun.io,
env.SHLVL=1, file.separator=/, java.specification.name=Java Platform API
Specification, java.class.version=50.0, user.country=EN,
java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home,
java.vm.info=mixed mode, env.XML_CATALOG_FILES=/sw/etc/xml/catalog,
env.LOGNAME=elif, os.version=10.6.3, env.TERM_PROGRAM_VERSION=273,
path.separator=:, java.vm.version=14.3-b01-101,
java.awt.printerjob=apple.awt.CPrinterJob, env.TERM=xterm-color,
sun.io.unicode.encoding=UnicodeLittle, awt.toolkit=apple.awt.CToolkit,
socksNonProxyHosts=local|*.local|169.254/16|*.169.254/16,
ftp.nonProxyHosts=local|*.local|169.254/16|*.169.254/16,
env.MANPATH=/sw/share/man:/opt/local/share/man:/Library/Frameworks/Python.framework/Versions/2.6/share/man:/usr/share/man:/usr/X11/man:/sw/lib/perl5/5.10.0/man:/usr/X11R6/man,
user.home=/Users/elif,
env.OLDPWD=/Users/elif/DevelopmentLibraries/tika/svn/tika-site,
java.specification.vendor=Sun Microsystems Inc.,
env.TERM_PROGRAM=Apple_Terminal,
java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java,
java.vendor.url=http://www.apple.com/, distMgmtSnapshotsName=Apache
Development Snapshot Repository, java.vm.vendor=Apple Inc.,
gopherProxySet=false, maven.home=/usr/share/maven, java.runtime.name=Java(TM)
SE Runtime Environment,
java.class.path=/usr/share/maven/boot/classworlds-1.1.jar,
maven.compile.target=1.5, java.vm.specification.name=Java Virtual Machine
Specification, java.vm.specification.version=1.0,
sun.os.patch.level=unknown, sun.cpu.endian=little, env.HOME=/Users/elif,
env.LMILAUNCHAGENTFIXER_GOODGUISESSION=1,
java.io.tmpdir=/var/folders/jU/jUxoh2kOHfKUjXUZkhwDBU+++TI/-Tmp-/,
java.vendor.url.bug=http://bugreport.apple.com/,
env.SSH_AUTH_SOCK=/tmp/launch-s9lK4H/Listeners,
env.MAVEN_OPTS=-Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF8
-Duser.language=en -Duser.country=EN, distMgmtSnapshotsUrl=
https://repository.apache.org/content/repositories/snapshots,
env.COMMAND_MODE=unix2003, env.XDG_CONFIG_HOME=/Users/elif/.config,
mrj.build=10M3025, java.awt.graphicsenv=apple.awt.CGraphicsEnvironment,
os.arch=x86_64, env.PERL5LIB=/sw/lib/perl5:/sw/lib/perl5/darwin,
java.ext.dirs=/Library/Java/Extensions:/System/Library/Java/Extensions:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/ext,
user.dir=/Users/elif/DevelopmentLibraries/tika/svn/tika-site,
env.ac_cv_path_INTLTOOL_PERL=/usr/bin/perl, mrj.version=1060.1.6.0_17-248,
env.INFOPATH=/sw/share/info:/sw/info:/usr/share/info, line.separator=
, java.vm.name=Java HotSpot(TM) 64-Bit Server VM,
env.SGML_CATALOG_FILES=/sw/etc/sgml/catalog,
env.XDG_CACHE_HOME=/Users/elif/.cache, file.encoding=UTF8,
java.specification.version=1.6, env.XDG_CONFIG_DIRS=/sw/etc/xdg,
env.XDG_DATA_HOME=/Users/elif/.local/share}
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[DEBUG]   (f) encoding = UTF-8
[DEBUG]   (f) encoding = UTF-8
[DEBUG] properties used {env.VERSIONER_PERL_PREFER_32_BIT=yes,
java.vendor=Apple Inc., maven.compile.source=1.5,
sun.java.launcher=SUN_STANDARD, sun.management.compiler=HotSpot 64-Bit
Server Compiler, os.name=Mac OS X,
sun.boot.class.path=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jsfd.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/ui.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/laf.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/sunrsasign.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jsse.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/jce.jar:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Classes/charsets.jar,
env.TMPDIR=/var/folders/jU/jUxoh2kOHfKUjXUZkhwDBU+++TI/-Tmp-/,
env.PWD=/Users/elif/DevelopmentLibraries/tika/svn/tika-site,
env.GDK_USE_XFT=1, env.LANG=tr_TR.UTF-8, java.vm.specification.vendor=Sun
Microsystems Inc., java.runtime.version=1.6.0_17-b04-248-10M3025,
env.Apple_PubSub_Socket_Render=/tmp/launch-4tCNW2/Render,
project.build.sourceEncoding=UTF-8, env.DISPLAY=/tmp/launch-0t3tnt/org.x:0,
user.name=elif, env.JAVA_MAIN_CLASS_49968=org.codehaus.classworlds.Launcher,
env.USER=elif, env.SHELL=/bin/bash, env.__CF_USER_TEXT_ENCODING=0x1F5:35:24,
awt.nativeDoubleBuffering=true, env.XDG_DATA_DIRS=/sw/share,
env.PATH=/sw/bin:/sw/sbin:/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/X11R6/bin,
user.language=en,
sun.boot.library.path=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Libraries,
classworlds.conf=/usr/share/maven/bin/m2.conf, java.version=1.6.0_17,
user.timezone=Europe/Istanbul, organization.logo=
http://www.apache.org/images/asf_logo_wide.gif, sun.arch.data.model=64,
http.nonProxyHosts=local|*.local|169.254/16|*.169.254/16,
java.endorsed.dirs=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/endorsed,
sun.cpu.isalist=, sun.jnu.encoding=UTF-8, file.encoding.pkg=sun.io,
env.SHLVL=1, file.separator=/, java.specification.name=Java Platform API
Specification, java.class.version=50.0, user.country=EN,
java.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home,
java.vm.info=mixed mode, env.XML_CATALOG_FILES=/sw/etc/xml/catalog,
env.LOGNAME=elif, os.version=10.6.3, env.TERM_PROGRAM_VERSION=273,
path.separator=:, java.vm.version=14.3-b01-101,
java.awt.printerjob=apple.awt.CPrinterJob, env.TERM=xterm-color,
sun.io.unicode.encoding=UnicodeLittle, awt.toolkit=apple.awt.CToolkit,
socksNonProxyHosts=local|*.local|169.254/16|*.169.254/16,
ftp.nonProxyHosts=local|*.local|169.254/16|*.169.254/16,
env.MANPATH=/sw/share/man:/opt/local/share/man:/Library/Frameworks/Python.framework/Versions/2.6/share/man:/usr/share/man:/usr/X11/man:/sw/lib/perl5/5.10.0/man:/usr/X11R6/man,
user.home=/Users/elif,
env.OLDPWD=/Users/elif/DevelopmentLibraries/tika/svn/tika-site,
java.specification.vendor=Sun Microsystems Inc.,
env.TERM_PROGRAM=Apple_Terminal,
java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java,
java.vendor.url=http://www.apple.com/, distMgmtSnapshotsName=Apache
Development Snapshot Repository, java.vm.vendor=Apple Inc.,
gopherProxySet=false, maven.home=/usr/share/maven, java.runtime.name=Java(TM)
SE Runtime Environment,
java.class.path=/usr/share/maven/boot/classworlds-1.1.jar,
maven.compile.target=1.5, java.vm.specification.name=Java Virtual Machine
Specification, java.vm.specification.version=1.0,
sun.os.patch.level=unknown, sun.cpu.endian=little, env.HOME=/Users/elif,
env.LMILAUNCHAGENTFIXER_GOODGUISESSION=1,
java.io.tmpdir=/var/folders/jU/jUxoh2kOHfKUjXUZkhwDBU+++TI/-Tmp-/,
java.vendor.url.bug=http://bugreport.apple.com/,
env.SSH_AUTH_SOCK=/tmp/launch-s9lK4H/Listeners,
env.MAVEN_OPTS=-Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF8
-Duser.language=en -Duser.country=EN, distMgmtSnapshotsUrl=
https://repository.apache.org/content/repositories/snapshots,
env.COMMAND_MODE=unix2003, env.XDG_CONFIG_HOME=/Users/elif/.config,
mrj.build=10M3025, java.awt.graphicsenv=apple.awt.CGraphicsEnvironment,
os.arch=x86_64, env.PERL5LIB=/sw/lib/perl5:/sw/lib/perl5/darwin,
java.ext.dirs=/Library/Java/Extensions:/System/Library/Java/Extensions:/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/ext,
user.dir=/Users/elif/DevelopmentLibraries/tika/svn/tika-site,
env.ac_cv_path_INTLTOOL_PERL=/usr/bin/perl, mrj.version=1060.1.6.0_17-248,
env.INFOPATH=/sw/share/info:/sw/info:/usr/share/info, line.separator=
, java.vm.name=Java HotSpot(TM) 64-Bit Server VM,
env.SGML_CATALOG_FILES=/sw/etc/sgml/catalog,
env.XDG_CACHE_HOME=/Users/elif/.cache, file.encoding=UTF8,
java.specification.version=1.6, env.XDG_CONFIG_DIRS=/sw/etc/xdg,
env.XDG_DATA_HOME=/Users/elif/.local/share}
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[DEBUG]   (f) encoding = UTF-8


On Mon, Apr 12, 2010 at 5:10 PM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> On Mon, Apr 12, 2010 at 3:57 PM, Elif T. Kus <el...@gmail.com> wrote:
> > I checked out from svn. Ran "maven install" and got the same errors.
>
> Can you try adding the -X option to the Maven run: mvn -X clean install
>
> The (quite verbose) output should contain a section like this:
>
> [DEBUG] Configuring mojo
> 'org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile' -->
> [DEBUG]   (f) basedir = /home/jukka/src/tika-0.7/tika-core
> [DEBUG]   (f) buildDirectory = /home/jukka/src/tika-0.7/tika-core/target
> [DEBUG]   (f) classpathElements =
> [/home/jukka/src/tika-0.7/tika-core/target/classes]
> [DEBUG]   (f) compileSourceRoots =
> [/home/jukka/src/tika-0.7/tika-core/src/main/java]
> [DEBUG]   (f) compilerId = javac
> [DEBUG]   (f) debug = true
> [DEBUG]   (f) encoding = UTF-8
> [DEBUG]   (f) failOnError = true
> [DEBUG]   (f) fork = false
> [DEBUG]   (f) optimize = false
> [DEBUG]   (f) outputDirectory =
> /home/jukka/src/tika-0.7/tika-core/target/classes
> [DEBUG]   (f) outputFileName = tika-core-0.7
> [DEBUG]   (f) projectArtifact = org.apache.tika:tika-core:bundle:0.7
> [DEBUG]   (f) showDeprecation = false
> [DEBUG]   (f) showWarnings = false
> [DEBUG]   (f) source = 1.5
> [DEBUG]   (f) staleMillis = 0
> [DEBUG]   (f) target = 1.5
> [DEBUG]   (f) verbose = false
> [DEBUG] -- end configuration --
>
> Check that the "encoding = UTF-8" setting is included in the output you
> see.
>
> You may also want to try adding the <fork>true</fork> option to the
> maven-compiler-plugin configuration in the tika-parent POM.
>
> BR,
>
> Jukka Zitting
>



-- 
-------------------------------
www.islerburada.com
İş arama motoru

Re: Problem with encoding during installation on mac os x

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Mon, Apr 12, 2010 at 3:57 PM, Elif T. Kus <el...@gmail.com> wrote:
> I checked out from svn. Ran "maven install" and got the same errors.

Can you try adding the -X option to the Maven run: mvn -X clean install

The (quite verbose) output should contain a section like this:

[DEBUG] Configuring mojo
'org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile' -->
[DEBUG]   (f) basedir = /home/jukka/src/tika-0.7/tika-core
[DEBUG]   (f) buildDirectory = /home/jukka/src/tika-0.7/tika-core/target
[DEBUG]   (f) classpathElements =
[/home/jukka/src/tika-0.7/tika-core/target/classes]
[DEBUG]   (f) compileSourceRoots =
[/home/jukka/src/tika-0.7/tika-core/src/main/java]
[DEBUG]   (f) compilerId = javac
[DEBUG]   (f) debug = true
[DEBUG]   (f) encoding = UTF-8
[DEBUG]   (f) failOnError = true
[DEBUG]   (f) fork = false
[DEBUG]   (f) optimize = false
[DEBUG]   (f) outputDirectory =
/home/jukka/src/tika-0.7/tika-core/target/classes
[DEBUG]   (f) outputFileName = tika-core-0.7
[DEBUG]   (f) projectArtifact = org.apache.tika:tika-core:bundle:0.7
[DEBUG]   (f) showDeprecation = false
[DEBUG]   (f) showWarnings = false
[DEBUG]   (f) source = 1.5
[DEBUG]   (f) staleMillis = 0
[DEBUG]   (f) target = 1.5
[DEBUG]   (f) verbose = false
[DEBUG] -- end configuration --

Check that the "encoding = UTF-8" setting is included in the output you see.

You may also want to try adding the <fork>true</fork> option to the
maven-compiler-plugin configuration in the tika-parent POM.

BR,

Jukka Zitting

Re: Problem with encoding during installation on mac os x

Posted by "Elif T. Kus" <el...@gmail.com>.
I checked out from svn. Ran "maven install" and got the same errors.


On Mon, Apr 12, 2010 at 4:44 PM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> On Mon, Apr 12, 2010 at 3:01 PM, Elif T. Kus <el...@gmail.com> wrote:
> > The errors show that there's a problem with the lower case i.
> > [...]
> > What can I do?
> >
> > Failed tests:
> >   testDetect(org.apache.tika.detect.TypeDetectorTest)
> >   testLowerCase(org.apache.tika.mime.MediaTypeTest)
>
> Is it possible that you've used some editor to open and save some of
> the .java files in Tika before running the build? Our Maven build
> should automatically set the source file encoding to UTF-8 to prevent
> issues like this, but that won't help if the source files themselves
> have been modified.
>
> Can you reproduce the problem with a direct svn checkout?
>
> Jukka Zitting
>



-- 
-------------------------------
www.islerburada.com
İş arama motoru

Re: Problem with encoding during installation on mac os x

Posted by "Elif T. Kus" <el...@gmail.com>.
I downloaded the code and created the project in Eclipse. I didn't save the
files. Moreover I checked whether the encoding of the files was correct. I
checked several they looked ok (UTF-8).

(I tried both command line mvn install and through the eclipse plugin they
give the same errors.)

I'll do a svn direct checkout now.

Thank you for your quick answer,
Elif

On Mon, Apr 12, 2010 at 4:44 PM, Jukka Zitting <ju...@gmail.com>wrote:

> Hi,
>
> On Mon, Apr 12, 2010 at 3:01 PM, Elif T. Kus <el...@gmail.com> wrote:
> > The errors show that there's a problem with the lower case i.
> > [...]
> > What can I do?
> >
> > Failed tests:
> >   testDetect(org.apache.tika.detect.TypeDetectorTest)
> >   testLowerCase(org.apache.tika.mime.MediaTypeTest)
>
> Is it possible that you've used some editor to open and save some of
> the .java files in Tika before running the build? Our Maven build
> should automatically set the source file encoding to UTF-8 to prevent
> issues like this, but that won't help if the source files themselves
> have been modified.
>
> Can you reproduce the problem with a direct svn checkout?
>
> Jukka Zitting
>



-- 
-------------------------------
www.islerburada.com
İş arama motoru

Re: Problem with encoding during installation on mac os x

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Mon, Apr 12, 2010 at 3:01 PM, Elif T. Kus <el...@gmail.com> wrote:
> The errors show that there's a problem with the lower case i.
> [...]
> What can I do?
>
> Failed tests:
>   testDetect(org.apache.tika.detect.TypeDetectorTest)
>   testLowerCase(org.apache.tika.mime.MediaTypeTest)

Is it possible that you've used some editor to open and save some of
the .java files in Tika before running the build? Our Maven build
should automatically set the source file encoding to UTF-8 to prevent
issues like this, but that won't help if the source files themselves
have been modified.

Can you reproduce the problem with a direct svn checkout?

Jukka Zitting