You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by George <ge...@gmail.com> on 2020/01/17 06:48:29 UTC
unpacking a column from a JSON structured msg, using stream processing
I have a topic with message as per:
{'file_name' : filename,
'line_number' : src_line_number,
'section' : vTag,
'line_data' : line_data
}
I want to unpack line_data into multiple columns based on position.
Can I do this via stream processing. the output will go onto a new
topic/stream.
G
--
You have the obligation to inform one honestly of the risk, and as a person
you are committed to educate yourself to the total risk in any activity!
Once informed & totally aware of the risk,
every fool has the right to kill or injure themselves as they see fit!
Re: unpacking a column from a JSON structured msg, using stream processing
Posted by George <ge...@gmail.com>.
ignore comment
just saw the kSQL button.
G
On Fri, Jan 17, 2020 at 1:43 PM George <ge...@gmail.com> wrote:
> Hi Bruno
>
> From the description this looks exactly whats needed, will have to see how
> to do this as I haven't touched Java in 20 yrs, someone did mention to me
> this is also possible via kSQL and a user defined method.
>
> Any chance anyone has a example?
>
> G
>
> On Fri, Jan 17, 2020 at 10:21 AM Bruno Cadonna <br...@confluent.io> wrote:
>
>> Hi George,
>>
>> Could the following tutorial help you?
>>
>> https://kafka-tutorials.confluent.io/transform-a-stream-of-events/kstreams.html
>>
>> Best,
>> Bruno
>>
>> On Fri, Jan 17, 2020 at 7:48 AM George <ge...@gmail.com> wrote:
>> >
>> > I have a topic with message as per:
>> >
>> > {'file_name' : filename,
>> > 'line_number' : src_line_number,
>> > 'section' : vTag,
>> > 'line_data' : line_data
>> > }
>> >
>> >
>> > I want to unpack line_data into multiple columns based on position.
>> >
>> > Can I do this via stream processing. the output will go onto a new
>> > topic/stream.
>> >
>> > G
>> >
>> > --
>> > You have the obligation to inform one honestly of the risk, and as a
>> person
>> > you are committed to educate yourself to the total risk in any activity!
>> >
>> > Once informed & totally aware of the risk,
>> > every fool has the right to kill or injure themselves as they see fit!
>>
>
>
> --
> You have the obligation to inform one honestly of the risk, and as a person
> you are committed to educate yourself to the total risk in any activity!
>
> Once informed & totally aware of the risk,
> every fool has the right to kill or injure themselves as they see fit!
>
--
You have the obligation to inform one honestly of the risk, and as a person
you are committed to educate yourself to the total risk in any activity!
Once informed & totally aware of the risk,
every fool has the right to kill or injure themselves as they see fit!
Re: unpacking a column from a JSON structured msg, using stream processing
Posted by George <ge...@gmail.com>.
Hi Bruno
From the description this looks exactly whats needed, will have to see how
to do this as I haven't touched Java in 20 yrs, someone did mention to me
this is also possible via kSQL and a user defined method.
Any chance anyone has a example?
G
On Fri, Jan 17, 2020 at 10:21 AM Bruno Cadonna <br...@confluent.io> wrote:
> Hi George,
>
> Could the following tutorial help you?
>
> https://kafka-tutorials.confluent.io/transform-a-stream-of-events/kstreams.html
>
> Best,
> Bruno
>
> On Fri, Jan 17, 2020 at 7:48 AM George <ge...@gmail.com> wrote:
> >
> > I have a topic with message as per:
> >
> > {'file_name' : filename,
> > 'line_number' : src_line_number,
> > 'section' : vTag,
> > 'line_data' : line_data
> > }
> >
> >
> > I want to unpack line_data into multiple columns based on position.
> >
> > Can I do this via stream processing. the output will go onto a new
> > topic/stream.
> >
> > G
> >
> > --
> > You have the obligation to inform one honestly of the risk, and as a
> person
> > you are committed to educate yourself to the total risk in any activity!
> >
> > Once informed & totally aware of the risk,
> > every fool has the right to kill or injure themselves as they see fit!
>
--
You have the obligation to inform one honestly of the risk, and as a person
you are committed to educate yourself to the total risk in any activity!
Once informed & totally aware of the risk,
every fool has the right to kill or injure themselves as they see fit!
Re: unpacking a column from a JSON structured msg, using stream processing
Posted by Bruno Cadonna <br...@confluent.io>.
Hi George,
Could the following tutorial help you?
https://kafka-tutorials.confluent.io/transform-a-stream-of-events/kstreams.html
Best,
Bruno
On Fri, Jan 17, 2020 at 7:48 AM George <ge...@gmail.com> wrote:
>
> I have a topic with message as per:
>
> {'file_name' : filename,
> 'line_number' : src_line_number,
> 'section' : vTag,
> 'line_data' : line_data
> }
>
>
> I want to unpack line_data into multiple columns based on position.
>
> Can I do this via stream processing. the output will go onto a new
> topic/stream.
>
> G
>
> --
> You have the obligation to inform one honestly of the risk, and as a person
> you are committed to educate yourself to the total risk in any activity!
>
> Once informed & totally aware of the risk,
> every fool has the right to kill or injure themselves as they see fit!