You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@wayang.apache.org by Jan Lukavský <je...@seznam.cz> on 2024/02/03 13:50:26 UTC
Hello to Wayang community
Hi Wayang community,
I got recently pointed out to your project and it got my attention. I'd
like to introduce myself - my name is Jan, I'm committer and PMC member
of Apache Beam project [1] and I also happen to work on a
streaming-first platform which might be a possible companion of Wayang.
The platform is called Proxima [2] and can be generally described as an
abstraction layer on top of various storages (streaming - e.g. Kafka,
random access - e.g. Cassandra, HBase, Elasitc and batch - e.g. HDFS,
S3, GCS) providing unified access for applications n terms of common
data schema that is applied across these storage types. The platform
replicates data with eventual consistency (replicating from streaming
storage - the commit-log), optionally providing ACID guarantees. It
seems to me it might make sense to build a connector that could extend
the capabilities of both Wayang and Proxima. Would here be someone
interesting in discussing this further?
Best wishes to the project!
Jan
[1] https://beam.apache.org/
[2] https://github.com/O2-Czech-Republic/proxima-platform
Re: Hello to Wayang community
Posted by Mirko Kämpf <mi...@gmail.com>.
Hi Jan,
our talk has been accepted, so I plan to attend in person.
Look forward meeting you in person there.
Cheers,
Mirko
Am Di., 5. März 2024 um 14:30 Uhr schrieb Jan Lukavský <je...@seznam.cz>:
> Hi Zoi,
>
> sorry for very late answer, your reaction somehow ended in my spam
> folder :-(.
>
> I'm currently sort of short on time, but is there any chance any of you
> will attend Community Over Code EU? We might discuss there in person. If
> not I will contact you by the end of April or so.
>
> Thanks for the offer!
>
> Jan
>
> On 2/5/24 10:23, Zoi Kaoudi wrote:
> > Hi Jan,
> > I would be happy to have a chat with you and explain you how adding new
> platforms is being done. Let me know your availabilities.
> >
> > Best
> > --
> > Zoi
> >
> > Στις Δευτέρα 5 Φεβρουαρίου 2024 στις 10:01:17 π.μ. CET, ο χρήστης
> Alexander Alten <al...@databloom.ai> έγραψε:
> >
> > Hi Jan,
> >
> > The Wayang-platforms subdirectory is a good pace for additional platform
> support:
> > https://github.com/apache/incubator-wayang/tree/main/wayang-platforms
> >
> > Mirko is working on a kafka integration, and a python API is also in the
> making. Kaustubh would be a good counterpart, he wrote the SQL integration
> using Apache Calcite.
> >
> > Best,
> > —Alex
> >
> >> On Feb 5, 2024, at 09:06, Jan Lukavský <je...@seznam.cz> wrote:
> >>
> >> Hi Alex,
> >>
> >> could you please provide some links to where to start looking into
> that? I'd like to better understand how exactly to implement a plugin and
> how to use it afterwards.
> >> Thanks!
> >> Jan
> >>
> >> On 2/3/24 17:39, Alexander Alten wrote:
> >>> Hi Jan,
> >>>
> >>> Welcome!
> >>>
> >>> Proxima looks good, and a plugin to Wayang would extend the
> capabilities into streaming data analytics and realtime ML / AI, kind of.
> Looking forward to contributions!
> >>>
> >>> Best,
> >>> —Alex
> >>> —
> >>> Alexander Alten
> >>> PPMC Apache Wayang
> >>> aloalt@apache.org
> >>>
> >>> @apachewayang
> >>>
> >>>> On Feb 3, 2024, at 14:50, Jan Lukavský <je...@seznam.cz> wrote:
> >>>>
> >>>> Hi Wayang community,
> >>>>
> >>>> I got recently pointed out to your project and it got my attention.
> I'd like to introduce myself - my name is Jan, I'm committer and PMC member
> of Apache Beam project [1] and I also happen to work on a streaming-first
> platform which might be a possible companion of Wayang. The platform is
> called Proxima [2] and can be generally described as an abstraction layer
> on top of various storages (streaming - e.g. Kafka, random access - e.g.
> Cassandra, HBase, Elasitc and batch - e.g. HDFS, S3, GCS) providing unified
> access for applications n terms of common data schema that is applied
> across these storage types. The platform replicates data with eventual
> consistency (replicating from streaming storage - the commit-log),
> optionally providing ACID guarantees. It seems to me it might make sense to
> build a connector that could extend the capabilities of both Wayang and
> Proxima. Would here be someone interesting in discussing this further?
> >>>>
> >>>> Best wishes to the project!
> >>>>
> >>>> Jan
> >>>>
> >>>> [1]
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://beam.apache.org/%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw2otR99KHMV-hq3-M4lbpxE&source=gmail-imap&ust=1707725192000000&usg=AOvVaw0P13FtP___jbQRV8TUlqCj
> >>>>
> >>>> [2]
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/O2-Czech-Republic/proxima-platform%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw1yWtH0q0oF241UepthR_zt&source=gmail-imap&ust=1707725192000000&usg=AOvVaw3f531w6sJBZOe8fOG5Yfu5
> >
>
--
Dr. rer. nat. Mirko Kämpf
Müchelner Str. 23
06259 Frankleben
Re: Hello to Wayang community
Posted by Jan Lukavský <je...@seznam.cz>.
Hi Zoi,
sorry for very late answer, your reaction somehow ended in my spam
folder :-(.
I'm currently sort of short on time, but is there any chance any of you
will attend Community Over Code EU? We might discuss there in person. If
not I will contact you by the end of April or so.
Thanks for the offer!
Jan
On 2/5/24 10:23, Zoi Kaoudi wrote:
> Hi Jan,
> I would be happy to have a chat with you and explain you how adding new platforms is being done. Let me know your availabilities.
>
> Best
> --
> Zoi
>
> Στις Δευτέρα 5 Φεβρουαρίου 2024 στις 10:01:17 π.μ. CET, ο χρήστης Alexander Alten <al...@databloom.ai> έγραψε:
>
> Hi Jan,
>
> The Wayang-platforms subdirectory is a good pace for additional platform support:
> https://github.com/apache/incubator-wayang/tree/main/wayang-platforms
>
> Mirko is working on a kafka integration, and a python API is also in the making. Kaustubh would be a good counterpart, he wrote the SQL integration using Apache Calcite.
>
> Best,
> —Alex
>
>> On Feb 5, 2024, at 09:06, Jan Lukavský <je...@seznam.cz> wrote:
>>
>> Hi Alex,
>>
>> could you please provide some links to where to start looking into that? I'd like to better understand how exactly to implement a plugin and how to use it afterwards.
>> Thanks!
>> Jan
>>
>> On 2/3/24 17:39, Alexander Alten wrote:
>>> Hi Jan,
>>>
>>> Welcome!
>>>
>>> Proxima looks good, and a plugin to Wayang would extend the capabilities into streaming data analytics and realtime ML / AI, kind of. Looking forward to contributions!
>>>
>>> Best,
>>> —Alex
>>> —
>>> Alexander Alten
>>> PPMC Apache Wayang
>>> aloalt@apache.org
>>>
>>> @apachewayang
>>>
>>>> On Feb 3, 2024, at 14:50, Jan Lukavský <je...@seznam.cz> wrote:
>>>>
>>>> Hi Wayang community,
>>>>
>>>> I got recently pointed out to your project and it got my attention. I'd like to introduce myself - my name is Jan, I'm committer and PMC member of Apache Beam project [1] and I also happen to work on a streaming-first platform which might be a possible companion of Wayang. The platform is called Proxima [2] and can be generally described as an abstraction layer on top of various storages (streaming - e.g. Kafka, random access - e.g. Cassandra, HBase, Elasitc and batch - e.g. HDFS, S3, GCS) providing unified access for applications n terms of common data schema that is applied across these storage types. The platform replicates data with eventual consistency (replicating from streaming storage - the commit-log), optionally providing ACID guarantees. It seems to me it might make sense to build a connector that could extend the capabilities of both Wayang and Proxima. Would here be someone interesting in discussing this further?
>>>>
>>>> Best wishes to the project!
>>>>
>>>> Jan
>>>>
>>>> [1] https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://beam.apache.org/%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw2otR99KHMV-hq3-M4lbpxE&source=gmail-imap&ust=1707725192000000&usg=AOvVaw0P13FtP___jbQRV8TUlqCj
>>>>
>>>> [2] https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/O2-Czech-Republic/proxima-platform%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw1yWtH0q0oF241UepthR_zt&source=gmail-imap&ust=1707725192000000&usg=AOvVaw3f531w6sJBZOe8fOG5Yfu5
>
Re: Hello to Wayang community
Posted by Zoi Kaoudi <zk...@yahoo.gr.INVALID>.
Hi Jan,
I would be happy to have a chat with you and explain you how adding new platforms is being done. Let me know your availabilities.
Best
--
Zoi
Στις Δευτέρα 5 Φεβρουαρίου 2024 στις 10:01:17 π.μ. CET, ο χρήστης Alexander Alten <al...@databloom.ai> έγραψε:
Hi Jan,
The Wayang-platforms subdirectory is a good pace for additional platform support:
https://github.com/apache/incubator-wayang/tree/main/wayang-platforms
Mirko is working on a kafka integration, and a python API is also in the making. Kaustubh would be a good counterpart, he wrote the SQL integration using Apache Calcite.
Best,
—Alex
> On Feb 5, 2024, at 09:06, Jan Lukavský <je...@seznam.cz> wrote:
>
> Hi Alex,
>
> could you please provide some links to where to start looking into that? I'd like to better understand how exactly to implement a plugin and how to use it afterwards.
> Thanks!
> Jan
>
> On 2/3/24 17:39, Alexander Alten wrote:
>> Hi Jan,
>>
>> Welcome!
>>
>> Proxima looks good, and a plugin to Wayang would extend the capabilities into streaming data analytics and realtime ML / AI, kind of. Looking forward to contributions!
>>
>> Best,
>> —Alex
>> —
>> Alexander Alten
>> PPMC Apache Wayang
>> aloalt@apache.org
>>
>> @apachewayang
>>
>>> On Feb 3, 2024, at 14:50, Jan Lukavský <je...@seznam.cz> wrote:
>>>
>>> Hi Wayang community,
>>>
>>> I got recently pointed out to your project and it got my attention. I'd like to introduce myself - my name is Jan, I'm committer and PMC member of Apache Beam project [1] and I also happen to work on a streaming-first platform which might be a possible companion of Wayang. The platform is called Proxima [2] and can be generally described as an abstraction layer on top of various storages (streaming - e.g. Kafka, random access - e.g. Cassandra, HBase, Elasitc and batch - e.g. HDFS, S3, GCS) providing unified access for applications n terms of common data schema that is applied across these storage types. The platform replicates data with eventual consistency (replicating from streaming storage - the commit-log), optionally providing ACID guarantees. It seems to me it might make sense to build a connector that could extend the capabilities of both Wayang and Proxima. Would here be someone interesting in discussing this further?
>>>
>>> Best wishes to the project!
>>>
>>> Jan
>>>
>>> [1] https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://beam.apache.org/%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw2otR99KHMV-hq3-M4lbpxE&source=gmail-imap&ust=1707725192000000&usg=AOvVaw0P13FtP___jbQRV8TUlqCj
>>>
>>> [2] https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/O2-Czech-Republic/proxima-platform%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw1yWtH0q0oF241UepthR_zt&source=gmail-imap&ust=1707725192000000&usg=AOvVaw3f531w6sJBZOe8fOG5Yfu5
--
databloom AI, Inc.
3401 N. MIAMI AVE. STE 230
33127 Miami, Florida
United States
www.databloom.ai <https://www.databloom.ai>
-- Please
consider the environment before printing this email --
Disclaimer:
The
content of this message is confidential. If you have received it by
mistake, please inform us by an email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone. The integrity and security of this email cannot be
guaranteed over the Internet. Therefore, the sender will not be held liable
for any damage caused by the message.
Re: Hello to Wayang community
Posted by Alexander Alten <al...@databloom.ai>.
Hi Jan,
The Wayang-platforms subdirectory is a good pace for additional platform support:
https://github.com/apache/incubator-wayang/tree/main/wayang-platforms
Mirko is working on a kafka integration, and a python API is also in the making. Kaustubh would be a good counterpart, he wrote the SQL integration using Apache Calcite.
Best,
—Alex
> On Feb 5, 2024, at 09:06, Jan Lukavský <je...@seznam.cz> wrote:
>
> Hi Alex,
>
> could you please provide some links to where to start looking into that? I'd like to better understand how exactly to implement a plugin and how to use it afterwards.
> Thanks!
> Jan
>
> On 2/3/24 17:39, Alexander Alten wrote:
>> Hi Jan,
>>
>> Welcome!
>>
>> Proxima looks good, and a plugin to Wayang would extend the capabilities into streaming data analytics and realtime ML / AI, kind of. Looking forward to contributions!
>>
>> Best,
>> —Alex
>> —
>> Alexander Alten
>> PPMC Apache Wayang
>> aloalt@apache.org
>>
>> @apachewayang
>>
>>> On Feb 3, 2024, at 14:50, Jan Lukavský <je...@seznam.cz> wrote:
>>>
>>> Hi Wayang community,
>>>
>>> I got recently pointed out to your project and it got my attention. I'd like to introduce myself - my name is Jan, I'm committer and PMC member of Apache Beam project [1] and I also happen to work on a streaming-first platform which might be a possible companion of Wayang. The platform is called Proxima [2] and can be generally described as an abstraction layer on top of various storages (streaming - e.g. Kafka, random access - e.g. Cassandra, HBase, Elasitc and batch - e.g. HDFS, S3, GCS) providing unified access for applications n terms of common data schema that is applied across these storage types. The platform replicates data with eventual consistency (replicating from streaming storage - the commit-log), optionally providing ACID guarantees. It seems to me it might make sense to build a connector that could extend the capabilities of both Wayang and Proxima. Would here be someone interesting in discussing this further?
>>>
>>> Best wishes to the project!
>>>
>>> Jan
>>>
>>> [1] https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://beam.apache.org/%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw2otR99KHMV-hq3-M4lbpxE&source=gmail-imap&ust=1707725192000000&usg=AOvVaw0P13FtP___jbQRV8TUlqCj
>>>
>>> [2] https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/O2-Czech-Republic/proxima-platform%26source%3Dgmail-imap%26ust%3D1707573037000000%26usg%3DAOvVaw1yWtH0q0oF241UepthR_zt&source=gmail-imap&ust=1707725192000000&usg=AOvVaw3f531w6sJBZOe8fOG5Yfu5
--
databloom AI, Inc.
3401 N. MIAMI AVE. STE 230
33127 Miami, Florida
United States
www.databloom.ai <https://www.databloom.ai>
-- Please
consider the environment before printing this email --
Disclaimer:
The
content of this message is confidential. If you have received it by
mistake, please inform us by an email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone. The integrity and security of this email cannot be
guaranteed over the Internet. Therefore, the sender will not be held liable
for any damage caused by the message.
Re: Hello to Wayang community
Posted by Jan Lukavský <je...@seznam.cz>.
Hi Alex,
could you please provide some links to where to start looking into that?
I'd like to better understand how exactly to implement a plugin and how
to use it afterwards.
Thanks!
Jan
On 2/3/24 17:39, Alexander Alten wrote:
> Hi Jan,
>
> Welcome!
>
> Proxima looks good, and a plugin to Wayang would extend the capabilities into streaming data analytics and realtime ML / AI, kind of. Looking forward to contributions!
>
> Best,
> —Alex
>
> —
> Alexander Alten
> PPMC Apache Wayang
> aloalt@apache.org
>
> @apachewayang
>
>> On Feb 3, 2024, at 14:50, Jan Lukavský <je...@seznam.cz> wrote:
>>
>> Hi Wayang community,
>>
>> I got recently pointed out to your project and it got my attention. I'd like to introduce myself - my name is Jan, I'm committer and PMC member of Apache Beam project [1] and I also happen to work on a streaming-first platform which might be a possible companion of Wayang. The platform is called Proxima [2] and can be generally described as an abstraction layer on top of various storages (streaming - e.g. Kafka, random access - e.g. Cassandra, HBase, Elasitc and batch - e.g. HDFS, S3, GCS) providing unified access for applications n terms of common data schema that is applied across these storage types. The platform replicates data with eventual consistency (replicating from streaming storage - the commit-log), optionally providing ACID guarantees. It seems to me it might make sense to build a connector that could extend the capabilities of both Wayang and Proxima. Would here be someone interesting in discussing this further?
>>
>> Best wishes to the project!
>>
>> Jan
>>
>> [1] https://www.google.com/url?q=https://beam.apache.org/&source=gmail-imap&ust=1707573037000000&usg=AOvVaw2otR99KHMV-hq3-M4lbpxE
>>
>> [2] https://www.google.com/url?q=https://github.com/O2-Czech-Republic/proxima-platform&source=gmail-imap&ust=1707573037000000&usg=AOvVaw1yWtH0q0oF241UepthR_zt
>>
>
Re: Hello to Wayang community
Posted by Alexander Alten <al...@databloom.ai>.
Hi Jan,
Welcome!
Proxima looks good, and a plugin to Wayang would extend the capabilities into streaming data analytics and realtime ML / AI, kind of. Looking forward to contributions!
Best,
—Alex
—
Alexander Alten
PPMC Apache Wayang
aloalt@apache.org
@apachewayang
> On Feb 3, 2024, at 14:50, Jan Lukavský <je...@seznam.cz> wrote:
>
> Hi Wayang community,
>
> I got recently pointed out to your project and it got my attention. I'd like to introduce myself - my name is Jan, I'm committer and PMC member of Apache Beam project [1] and I also happen to work on a streaming-first platform which might be a possible companion of Wayang. The platform is called Proxima [2] and can be generally described as an abstraction layer on top of various storages (streaming - e.g. Kafka, random access - e.g. Cassandra, HBase, Elasitc and batch - e.g. HDFS, S3, GCS) providing unified access for applications n terms of common data schema that is applied across these storage types. The platform replicates data with eventual consistency (replicating from streaming storage - the commit-log), optionally providing ACID guarantees. It seems to me it might make sense to build a connector that could extend the capabilities of both Wayang and Proxima. Would here be someone interesting in discussing this further?
>
> Best wishes to the project!
>
> Jan
>
> [1] https://www.google.com/url?q=https://beam.apache.org/&source=gmail-imap&ust=1707573037000000&usg=AOvVaw2otR99KHMV-hq3-M4lbpxE
>
> [2] https://www.google.com/url?q=https://github.com/O2-Czech-Republic/proxima-platform&source=gmail-imap&ust=1707573037000000&usg=AOvVaw1yWtH0q0oF241UepthR_zt
>
--
databloom AI, Inc.
3401 N. MIAMI AVE. STE 230
33127 Miami, Florida
United States
www.databloom.ai <https://www.databloom.ai>
-- Please
consider the environment before printing this email --
Disclaimer:
The
content of this message is confidential. If you have received it by
mistake, please inform us by an email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone. The integrity and security of this email cannot be
guaranteed over the Internet. Therefore, the sender will not be held liable
for any damage caused by the message.