You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by James Aley <ja...@swiftkey.com> on 2015/09/07 10:13:28 UTC
Java UDFs in GROUP BY expressions
Hi everyone,
I raised this JIRA ticket back in July:
https://issues.apache.org/jira/browse/SPARK-9435
The problem is that it seems Spark SQL doesn't recognise columns we
transform with a UDF when referenced in the GROUP BY clause. There's a
minimal reproduction Java file attached to illustrate the issue.
The equivalent code from Scala seems to work fine for me. Is anyone else
seeing this problem? For us, the attached code fails every time on Spark
1.4.1
Thanks,
James