You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Will Benton <wi...@redhat.com> on 2014/07/09 04:58:11 UTC

odd test suite failures while adding functions to Catalyst

Hi all,

I was testing an addition to Catalyst today (reimplementing a Hive UDF) and ran into some odd failures in the test suite.  In particular, it seems that what most of these have in common is that an array is spuriously reversed somewhere.  For example, the stddev tests in the HiveCompatibilitySuite all failed this way (note reversed synonyms list; note also that stddev isn't the function I reimplemented):

   [info] - udf_std *** FAILED ***
   [info]   Results do not match for udf_std:
   [info]   DESCRIBE FUNCTION EXTENDED std
   [info]   == Logical Plan ==
   [info]   NativeCommand DESCRIBE FUNCTION EXTENDED std
   [info]   
   [info]   == Optimized Logical Plan ==
   [info]   NativeCommand DESCRIBE FUNCTION EXTENDED std
   [info]   
   [info]   == Physical Plan ==
   [info]   NativeCommand DESCRIBE FUNCTION EXTENDED std, [result#38637:0]
   [info]   result
   [info]   !== HIVE - 2 row(s) ==                                         == CATALYST - 2 row(s) ==
   [info]    std(x) - Returns the standard deviation of a set of numbers   std(x) - Returns the standard deviation of a set of numbers
   [info]   !Synonyms: stddev_pop, stddev                                  Synonyms: stddev, stddev_pop (HiveComparisonTest.scala:372)

I also saw a reversed array (relative to the expected array) of Hive settings in HiveQuerySuite.

I'll probably be able to track down where things are going wrong after getting away from my desk for a bit, but I thought I'd send it out to the dev list in case this looks familiar to anyone.  Has anyone seen this kind of failure before?



thanks,
wb