DataFrame rollup with alias?

[Isabelle Phan <nl...@gmail.com>]

Hello,

I am new to Spark and just running some tests to get familiar with the APIs.

When calling the rollup function on my DataFrame, I get different results
when I alias the columns I am grouping on (see below for example data set).
I was expecting alias function to only affect the column name. Why is it
also affecting the rollup results?
(I know I can rename my columns after the rollup call, using
withColumnRenamed function, my question is just to get better understanding
of alias function.)

scala> df.show
+----+------+-----+
|Name|  Game|Score|
+----+------+-----+
| Bob|Game 1|   20|
| Bob|Game 2|   30|
| Lea|Game 1|   25|
| Lea|Game 2|   30|
| Ben|Game 1|    5|
| Ben|Game 3|   35|
| Bob|Game 3|   15|
+----+------+-----+

//rollup results as expected
scala> df.rollup(df("Name"), df("Game")).sum().orderBy("Name", "Game").show
+----+------+----------+
|Name|  Game|SUM(Score)|
+----+------+----------+
|null|  null|       160|
| Ben|  null|        40|
| Ben|Game 1|         5|
| Ben|Game 3|        35|
| Bob|  null|        65|
| Bob|Game 1|        20|
| Bob|Game 2|        30|
| Bob|Game 3|        15|
| Lea|  null|        55|
| Lea|Game 1|        25|
| Lea|Game 2|        30|
+----+------+----------+

//rollup with aliases return strange results
scala> df.rollup(df("Name") as "Player", df("Game") as
"Round").sum().orderBy("Player", "Round").show
+------+------+----------+
|Player| Round|SUM(Score)|
+------+------+----------+
|   Ben|Game 1|         5|
|   Ben|Game 1|         5|
|   Ben|Game 1|         5|
|   Ben|Game 3|        35|
|   Ben|Game 3|        35|
|   Ben|Game 3|        35|
|   Bob|Game 1|        20|
|   Bob|Game 1|        20|
|   Bob|Game 1|        20|
|   Bob|Game 2|        30|
|   Bob|Game 2|        30|
|   Bob|Game 2|        30|
|   Bob|Game 3|        15|
|   Bob|Game 3|        15|
|   Bob|Game 3|        15|
|   Lea|Game 1|        25|
|   Lea|Game 1|        25|
|   Lea|Game 1|        25|
|   Lea|Game 2|        30|
|   Lea|Game 2|        30|
+------+------+----------+


Thanks in advance for your help,

Isabelle