Applying schema dynamically in dataframe

[rahul c <rc...@gmail.com>]

Hi dev,

Currently I have a scenario where I am reading the data from Kafka using
spark dataframe.

Multiple data sources ingest the data into kafka same topic.

My dataframe now has got json string from multiple sources. I have got key
column from Kafka which can be used to identify which source data value it
is.

In spark how can I now apply the schema dynamically using the key colum
value.

Any help would be appreciated.

Thanks and Regards
Rahul

Re: Applying schema dynamically in dataframe

[ZHANG Wei <we...@outlook.com>]

May I get a sample scenario to understand the requirement?

-- 
Cheers,
-z

On Sat, 16 May 2020 11:45:03 +0530
rahul c <rc...@gmail.com> wrote:

> Hi dev,
> 
> Currently I have a scenario where I am reading the data from Kafka using
> spark dataframe.
> 
> Multiple data sources ingest the data into kafka same topic.
> 
> My dataframe now has got json string from multiple sources. I have got key
> column from Kafka which can be used to identify which source data value it
> is.
> 
> In spark how can I now apply the schema dynamically using the key colum
> value.
> 
> Any help would be appreciated.
> 
> Thanks and Regards
> Rahul

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org