You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Andrew Holway <an...@otternetworks.de> on 2016/01/23 22:02:09 UTC

Concatenating tables

Is there a data frame operation to do this?

+---------+
| A B C D |
+---------+
| 1 2 3 4 |
| 5 6 7 8 |
+---------+
+---------+
| A B C D |
+---------+
| 3 5 6 8 |
| 0 0 0 0 |
+---------+
+---------+
| A B C D |
+---------+
| 8 8 8 8 |
| 1 1 1 1 |
+---------+

Concatenated together to make this.

+---------+
| A B C D |
+---------+
| 1 2 3 4 |
| 5 6 7 8 |
| 3 5 6 8 |
| 0 0 0 0 |
| 8 8 8 8 |
| 1 1 1 1 |
+---------+

Thanks,

Andrew

Re: Concatenating tables

Posted by Ted Yu <yu...@gmail.com>.
How about this operation :

   * Returns a new [[DataFrame]] containing union of rows in this frame and
another frame.
   * This is equivalent to `UNION ALL` in SQL.
   * @group dfops
   * @since 1.3.0
   */
  def unionAll(other: DataFrame): DataFrame = withPlan {

FYI

On Sat, Jan 23, 2016 at 1:02 PM, Andrew Holway <
andrew.holway@otternetworks.de> wrote:

> Is there a data frame operation to do this?
>
> +---------+
> | A B C D |
> +---------+
> | 1 2 3 4 |
> | 5 6 7 8 |
> +---------+
> +---------+
> | A B C D |
> +---------+
> | 3 5 6 8 |
> | 0 0 0 0 |
> +---------+
> +---------+
> | A B C D |
> +---------+
> | 8 8 8 8 |
> | 1 1 1 1 |
> +---------+
>
> Concatenated together to make this.
>
> +---------+
> | A B C D |
> +---------+
> | 1 2 3 4 |
> | 5 6 7 8 |
> | 3 5 6 8 |
> | 0 0 0 0 |
> | 8 8 8 8 |
> | 1 1 1 1 |
> +---------+
>
> Thanks,
>
> Andrew
>

Re: Concatenating tables

Posted by Deenar Toraskar <de...@gmail.com>.
On 23 Jan 2016 9:18 p.m., "Deenar Toraskar" <
deenar.toraskar@thinkreactive.co.uk> wrote:

> Df.UnionAll(df2).unionall (df3)
> On 23 Jan 2016 9:02 p.m., "Andrew Holway" <an...@otternetworks.de>
> wrote:
>
>> Is there a data frame operation to do this?
>>
>> +---------+
>> | A B C D |
>> +---------+
>> | 1 2 3 4 |
>> | 5 6 7 8 |
>> +---------+
>> +---------+
>> | A B C D |
>> +---------+
>> | 3 5 6 8 |
>> | 0 0 0 0 |
>> +---------+
>> +---------+
>> | A B C D |
>> +---------+
>> | 8 8 8 8 |
>> | 1 1 1 1 |
>> +---------+
>>
>> Concatenated together to make this.
>>
>> +---------+
>> | A B C D |
>> +---------+
>> | 1 2 3 4 |
>> | 5 6 7 8 |
>> | 3 5 6 8 |
>> | 0 0 0 0 |
>> | 8 8 8 8 |
>> | 1 1 1 1 |
>> +---------+
>>
>> Thanks,
>>
>> Andrew
>>
>