From C* to DataFrames with JSON

[Jean-Francois Gosselin <jf...@gmail.com>]

Hi all,

I'm struggling (Spark / Scala newbie) to create a DataFrame from a C* table
but also create a DataFrame from column with json .

e.g. From C* table

| id | jsonData                      |
======================
| 1 |  {"a": "123", "b": "xyz" } |
+--+-------------------------------+
| 2 |  {"a": "3", "b": "bar" }     |


to Spark DataFrame:

| id |  a   |  b   |
===========
| 1 | 123 | xyz |
+--+------+-----+
| 2 | 3     | bar |


I'm using Spark 1.6 .

Thanks


JF

Re: From C* to DataFrames with JSON

[Takeshi Yamamuro <li...@gmail.com>]

If you upgrade to v2.1, you can use to_json/from_json in sql.functions.

On Fri, Feb 10, 2017 at 3:12 PM, Jean-Francois Gosselin <
jfgosselin@gmail.com> wrote:

>
> Hi all,
>
> I'm struggling (Spark / Scala newbie) to create a DataFrame from a C*
> table but also create a DataFrame from column with json .
>
> e.g. From C* table
>
> | id | jsonData                      |
> ======================
> | 1 |  {"a": "123", "b": "xyz" } |
> +--+-------------------------------+
> | 2 |  {"a": "3", "b": "bar" }     |
>
>
> to Spark DataFrame:
>
> | id |  a   |  b   |
> ===========
> | 1 | 123 | xyz |
> +--+------+-----+
> | 2 | 3     | bar |
>
>
> I'm using Spark 1.6 .
>
> Thanks
>
>
> JF
>



-- 
---
Takeshi Yamamuro