[jira] [Comment Edited] (PARQUET-1496) [Java] Update Scala to 2.12

["Zoltan Ivanfi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16801910#comment-16801910 ] 

Zoltan Ivanfi edited comment on PARQUET-1496 at 3/26/19 4:34 PM:
-----------------------------------------------------------------

There seems to be an unresolvable circular incompatibility issue here:
 * Java 11 is incompatible with Scala 2.10, needs newer version, like Scala 2.12
 * Scala 2.12 is incompatible with Scrooge 4, needs newer version, like Scrooge 19.
 * Scrooge 19 is incompatible with our {{parquet.thrift}} file for two reasons:
 ** It doesn't handle empty structs correctly. For further experimentation, this can be hacked around by changing each 
{noformat}
struct whatever {}
{noformat}
to
{noformat}
struct whatever {32767: optional i32 dummy;}
{noformat}
 ** It doesn't handle the {{String}} logical type correctly, because in the code it generates it does use fully qualified names. Since the name of this logical type shadows the stock String type, this leads to a compilation failure in the generated {{LogicalType.scala}} file.


was (Author: zi):
There seems to be an unresolvable circular incompatibility issue here:
 * Java 11 is incompatible with Scala 2.10, needs newer version, like Scala 2.12
 * Scala 2.12 is incompatible with Scrooge 4, needs newer version, like Scrooge 19.
 * Scrooge 19 is incompatible with our {{parquet.thrift}} file for two reasons:
 ** It doesn't handle empty structs correctly. For further experimentation, this can be hacked around by changing each {{struct whatever {}}} to {{struct whatever {32767: optional i32 dummy;}}}
 ** It doesn't handle the {{String}} logical type correctly, because in the code it generates it does use fully qualified names. Since the name of this logical type shadows the stock String type, this leads to a compilation failure in the generated {{LogicalType.scala}} file.

> [Java] Update Scala to 2.12
> ---------------------------
>
>                 Key: PARQUET-1496
>                 URL: https://issues.apache.org/jira/browse/PARQUET-1496
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: 1.10.0
>            Reporter: Uwe L. Korn
>            Assignee: Uwe L. Korn
>            Priority: Major
>              Labels: pull-request-available
>
> When trying to build the parquet-mr code on OSX Mojave with OpenJDK 10 and 9, the build fails for me in {{parquet-scala}} with:
> {code:java}
> [INFO] --- maven-scala-plugin:2.15.2:compile (default) @ parquet-scala_2.10 ---
> [INFO] Checking for multiple versions of scala
> [INFO] includes = [**/*.java,**/*.scala,]
> [INFO] excludes = []
> [INFO] /Users/uwe/tmp/apache-parquet-1.11.0/parquet-scala/src/main/scala:-1: info: compiling
> [INFO] Compiling 1 source files to /Users/uwe/tmp/apache-parquet-1.11.0/parquet-scala/target/classes at 1547922718010
> [ERROR] error: error while loading package, Missing dependency 'object java.lang.Object in compiler mirror', required by /Users/uwe/.m2/repository/org/scala-lang/scala-library/2.10.6/scala-library-2.10.6.jar(scala/package.class)
> [ERROR] error: error while loading package, Missing dependency 'object java.lang.Object in compiler mirror', required by /Users/uwe/.m2/repository/org/scala-lang/scala-library/2.10.6/scala-library-2.10.6.jar(scala/runtime/package.class)
> [ERROR] error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.
> [ERROR] at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
> [ERROR] at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
> [INFO] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
> [INFO] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
> [INFO] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
> [INFO] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
> [INFO] at scala.reflect.internal.Mirrors$RootsBase.getClassByName(Mirrors.scala:99)
> [INFO] at scala.reflect.internal.Mirrors$RootsBase.getRequiredClass(Mirrors.scala:102)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:264)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:264)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.AnyRefClass$lzycompute(Definitions.scala:263)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.AnyRefClass(Definitions.scala:263)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.specialPolyClass(Definitions.scala:1120)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.RepeatedParamClass$lzycompute(Definitions.scala:407)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.RepeatedParamClass(Definitions.scala:407)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses$lzycompute(Definitions.scala:1154)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses(Definitions.scala:1152)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1196)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1196)
> [INFO] at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1261)
> [INFO] at scala.tools.nsc.Global$Run.<init>(Global.scala:1290)
> [INFO] at scala.tools.nsc.Driver.doCompile(Driver.scala:32)
> [INFO] at scala.tools.nsc.Main$.doCompile(Main.scala:79)
> [INFO] at scala.tools.nsc.Driver.process(Driver.scala:54)
> [INFO] at scala.tools.nsc.Driver.main(Driver.scala:67)
> [INFO] at scala.tools.nsc.Main.main(Main.scala)
> [INFO] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [INFO] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [INFO] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [INFO] at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> [INFO] at org_scala_tools_maven_executions.MainHelper.runMain(MainHelper.java:161)
> [INFO] at org_scala_tools_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26){code}
> This is because the referenced JARs were made for a Scala version not supporting JDK 11, we need to update to 2.12.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)