You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Andrew Holway <an...@otternetworks.de> on 2016/01/23 22:02:09 UTC
Concatenating tables
Is there a data frame operation to do this?
+---------+
| A B C D |
+---------+
| 1 2 3 4 |
| 5 6 7 8 |
+---------+
+---------+
| A B C D |
+---------+
| 3 5 6 8 |
| 0 0 0 0 |
+---------+
+---------+
| A B C D |
+---------+
| 8 8 8 8 |
| 1 1 1 1 |
+---------+
Concatenated together to make this.
+---------+
| A B C D |
+---------+
| 1 2 3 4 |
| 5 6 7 8 |
| 3 5 6 8 |
| 0 0 0 0 |
| 8 8 8 8 |
| 1 1 1 1 |
+---------+
Thanks,
Andrew
Re: Concatenating tables
Posted by Ted Yu <yu...@gmail.com>.
How about this operation :
* Returns a new [[DataFrame]] containing union of rows in this frame and
another frame.
* This is equivalent to `UNION ALL` in SQL.
* @group dfops
* @since 1.3.0
*/
def unionAll(other: DataFrame): DataFrame = withPlan {
FYI
On Sat, Jan 23, 2016 at 1:02 PM, Andrew Holway <
andrew.holway@otternetworks.de> wrote:
> Is there a data frame operation to do this?
>
> +---------+
> | A B C D |
> +---------+
> | 1 2 3 4 |
> | 5 6 7 8 |
> +---------+
> +---------+
> | A B C D |
> +---------+
> | 3 5 6 8 |
> | 0 0 0 0 |
> +---------+
> +---------+
> | A B C D |
> +---------+
> | 8 8 8 8 |
> | 1 1 1 1 |
> +---------+
>
> Concatenated together to make this.
>
> +---------+
> | A B C D |
> +---------+
> | 1 2 3 4 |
> | 5 6 7 8 |
> | 3 5 6 8 |
> | 0 0 0 0 |
> | 8 8 8 8 |
> | 1 1 1 1 |
> +---------+
>
> Thanks,
>
> Andrew
>
Re: Concatenating tables
Posted by Deenar Toraskar <de...@gmail.com>.
On 23 Jan 2016 9:18 p.m., "Deenar Toraskar" <
deenar.toraskar@thinkreactive.co.uk> wrote:
> Df.UnionAll(df2).unionall (df3)
> On 23 Jan 2016 9:02 p.m., "Andrew Holway" <an...@otternetworks.de>
> wrote:
>
>> Is there a data frame operation to do this?
>>
>> +---------+
>> | A B C D |
>> +---------+
>> | 1 2 3 4 |
>> | 5 6 7 8 |
>> +---------+
>> +---------+
>> | A B C D |
>> +---------+
>> | 3 5 6 8 |
>> | 0 0 0 0 |
>> +---------+
>> +---------+
>> | A B C D |
>> +---------+
>> | 8 8 8 8 |
>> | 1 1 1 1 |
>> +---------+
>>
>> Concatenated together to make this.
>>
>> +---------+
>> | A B C D |
>> +---------+
>> | 1 2 3 4 |
>> | 5 6 7 8 |
>> | 3 5 6 8 |
>> | 0 0 0 0 |
>> | 8 8 8 8 |
>> | 1 1 1 1 |
>> +---------+
>>
>> Thanks,
>>
>> Andrew
>>
>