[jira] [Commented] (SAMZA-728) Samza job fails due to null pointer in JobCoordinator refreshJobModel

["Yan Fang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622984#comment-14622984 ] 

Yan Fang commented on SAMZA-728:
--------------------------------

+1 . Feel free to commit~~  Thanks. 

> Samza job fails due to null pointer in JobCoordinator refreshJobModel
> ---------------------------------------------------------------------
>
>                 Key: SAMZA-728
>                 URL: https://issues.apache.org/jira/browse/SAMZA-728
>             Project: Samza
>          Issue Type: Bug
>    Affects Versions: 0.10.0
>            Reporter: Navina Ramesh
>            Assignee: Navina Ramesh
>         Attachments: SAMZA-728-0.patch
>
>
> When I was running a simple Samza job, the job failed with the following NullPointerException:
> {code}
> Exception in thread "main" java.lang.NullPointerException
> 	at scala.collection.convert.Wrappers$MapWrapper$$anon$1$$anon$5$$anon$6.hashCode(Wrappers.scala:190)
> 	at java.util.AbstractMap.hashCode(AbstractMap.java:507)
> 	at java.util.Collections$UnmodifiableMap.hashCode(Collections.java:1492)
> 	at org.apache.samza.job.model.TaskModel.hashCode(TaskModel.java:96)
> 	at scala.runtime.ScalaRunTime$.hash(ScalaRunTime.scala:210)
> 	at scala.collection.immutable.HashSet.elemHashCode(HashSet.scala:65)
> 	at scala.collection.immutable.HashSet.computeHash(HashSet.scala:74)
> 	at scala.collection.immutable.HashSet.$plus(HashSet.scala:56)
> 	at scala.collection.immutable.HashSet.$plus(HashSet.scala:59)
> 	at scala.collection.immutable.Set$Set4.$plus(Set.scala:127)
> 	at scala.collection.immutable.Set$Set4.$plus(Set.scala:121)
> 	at scala.collection.mutable.SetBuilder.$plus$eq(SetBuilder.scala:24)
> 	at scala.collection.mutable.SetBuilder.$plus$eq(SetBuilder.scala:22)
> 	at scala.collection.generic.Growable$$anonfun$$plus$plus$eq$1.apply(Growable.scala:48)
> 	at scala.collection.generic.Growable$$anonfun$$plus$plus$eq$1.apply(Growable.scala:48)
> 	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> 	at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> 	at scala.collection.mutable.SetBuilder.$plus$plus$eq(SetBuilder.scala:22)
> 	at scala.collection.TraversableLike$class.to(TraversableLike.scala:629)
> 	at scala.collection.AbstractTraversable.to(Traversable.scala:105)
> 	at scala.collection.TraversableOnce$class.toSet(TraversableOnce.scala:267)
> 	at scala.collection.AbstractTraversable.toSet(Traversable.scala:105)
> 	at org.apache.samza.coordinator.JobCoordinator$.refreshJobModel(JobCoordinator.scala:262)
> 	at org.apache.samza.coordinator.JobCoordinator$.org$apache$samza$coordinator$JobCoordinator$$jobModelGenerator$1(JobCoordinator.scala:195)
> 	at org.apache.samza.coordinator.JobCoordinator$.initializeJobModel(JobCoordinator.scala:203)
> 	at org.apache.samza.coordinator.JobCoordinator$.getJobCoordinator(JobCoordinator.scala:90)
> 	at org.apache.samza.coordinator.JobCoordinator$.apply(JobCoordinator.scala:77)
> 	at org.apache.samza.job.yarn.SamzaAppMaster$.main(SamzaAppMaster.scala:77)
> 	at org.apache.samza.job.yarn.SamzaAppMaster.main(SamzaAppMaster.scala)
> {code}
> Strangely, I see this exception when there is more than 3 partitions in the input stream :( If the partition is <= 3, the job seems to run fine.
> Unrelated to the above situation, looks like we are bitten by this bug in Scala 2.10 where the "hashCode" implementation in the MapWrapper calls hashCode() on key and value without any null checks. More details here - https://github.com/scala/scala/pull/3704
> One option is to upgrade to Scala  > 2.10. But this may unleash a new set of problems. 
> The alternative is to use Java based maps whenever possible (even in Scala code). I am going with the second approach.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)