You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Yi Pan <ni...@gmail.com> on 2018/06/14 05:56:28 UTC
Announce of the next stream processing meetup @LinkedIn
Hi, all,
We have planed for some super-exciting talks at our next Streams Meetup on
July 19.
* Beam-Samza integration enabling new real time scenarios @ LinkedIn
* U-Replicator : Uber's multi-datacenter kafka mirroring service
* Concourse : LinkedIn’s near-real-time targeting and scoring platform for
notifications built on top of our ML and Stream Processing Infra.
You can sign up at https://lnkd.in/gz3WcWb
This time we will be open at both our Sunnyvale and San Francisco office.
Looking forward to see you!
Best,
-Yi
Re: Announce of the next stream processing meetup @LinkedIn
Posted by Yi Pan <ni...@gmail.com>.
Sorry. A correction to the email sent this morning: The date is July 19th
2018, as the meetup signup page says.
-Yi
On Wed, Jun 27, 2018 at 9:36 AM, Yi Pan <ni...@gmail.com> wrote:
> Just a reminder and some more details of the coming meetup, cheers!
>
> Hi Kafka, Brooklin and Samza Users,
> The Streams Infra team invites you to attend the Streams Processing meetup
> <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>
> on July 19th 2018 This meetup focuses on Apache Kafka, Apache Samza, and
> related streaming technologies. We will host the actual event at LinkedIn
> Sunnyvale office, and in addition to that, we will also host a "*viewing
> room*" from San Francisco.This time we have Xinyu Liu
> <https://www.linkedin.com/in/xinyu-liu-b0b21648/> from the Samza team
> talking about Apache Beam <https://beam.apache.org/> runner for Samza
> <https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/BEAM>. The Beam
> runner provides an ability to write-once but execute the same job in
> multiple environments (e.g. Hadoop for Batch Processing or Samza in
> Nearline). It also opens up possibilities for supporting different
> languages for stream processing (e.g. Python Applications on Samza). Our
> second speaker Hongliang Xu <https://www.linkedin.com/in/hongliangxu/> is
> from the Infrastructure team @Uber. His team recently built uReplicator to
> replicate data across Kafka clusters. You can find a blog
> <https://eng.uber.com/ureplicator/> about the original version of
> uReplicator here <https://eng.uber.com/ureplicator/> for reference. In
> his talk, Hongliang will focus on the new version of uReplicator, its
> architecture and share some of their learnings .Ajith Muralidharan
> <https://www.linkedin.com/in/ajithmuralidharan/> & Vivek Nelamangala
> <https://www.linkedin.com/in/viveknelamangala/> from LinkedIn will talk
> about how they built a near real time targeting and scoring platform
> (Concourse) for LinkedIn Notifications. Concourse is one of the largest
> Samza Jobs at LinkedIn and if you are building large scale streaming
> applications, this is the talk to attend.Below are some additional
> details about the talks. If you are interested to attend, Please RSVP via
> meetup.com
> <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>.
> You can also find additional details (streaming link, location, etc.) in
> the meetup link
> <https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>.
> Hope to see you there!
>
>
>
> *Location*:
>
> Main Event -
>
> Yosemite Conference Room, LinkedIn Corporate HQ in Sunnyvale.
>
> 2nd floor of 605 W Maude Ave, Sunnyvale, CA.
>
>
>
> Viewing Party -
>
> Lotta’s Fountain Conference Room, LinkedIn in San Francisco at 222 2nd
> Street, San Francisco, CA.
> Agenda:6PM: Doors open6-6:35 PM: Networking6:35-7:10 PM: Beam me up
> Samza: How we built a Samza Runner for Apache Beam (Speaker: Xinyu Liu,
> LinkedIn)
>
> Apache Beam provides an easy-to-use, and powerful model for
> state-of-the-art stream and batch processing, portability across a variety
> of languages, and the ability to converge offline and nearline data
> processing. At LinkedIn, we have developed a Samza Runner to leverage the
> cutting-edge features of Beam. This runner combines the large-scale
> streaming processing capabilities and first-class state support in Samza
> with the advancements in Beam data processing. In this talk, we will
> discuss the Beam API and its implementation in Samza and the benefits of
> Beam Runner to the Samza and Beam community.
> 7:15-7:50 PM: uReplicator: Uber Engineering’s Scalable Robust Kafka
> Replicator(Speaker: Hongliang Xu, Uber)
>
> At Uber, we operate 20+ Kafka clusters to collect system and application
> logs as well as event data from rider and driver apps. We need a Kafka
> replication solution to replicate data between Kafka clusters across
> multiple data centers for different purposes. This talk will introduce the
> history behind uReplicator and the high level architecture. As the original
> uReplicator ran into scalability challenges and operational overhead as the
> scale of Kafka clusters increased, we built the Federated uReplicator which
> addressed above issues and provide an extensible architecture for further
> scaling.
> 7:55-8:30 PM: Concourse - Near real time notifications platform at
> Linkedin (Speakers: Ajith Muralidharan & Vivek Nelamangala, LinkedIn)
>
> Concourse is LinkedIn’s first near-real-time targeting and scoring
> platform for notifications. In this talk, we will provide an in-depth
> overview of the design and discuss various scaling optimizations. We'll
> explain how Concourse can score millions of notifications per second, while
> supporting the use of feature-rich machine learning models based on
> terabytes of feature data.
> 8:30-9PM: Additional networking and Q&AThank you,Streams Infra @ LinkedIn
>
>
> On Wed, Jun 13, 2018 at 10:56 PM, Yi Pan <ni...@gmail.com> wrote:
>
>> Hi, all,
>>
>> We have planed for some super-exciting talks at our next Streams Meetup
>> on July 19.
>>
>> * Beam-Samza integration enabling new real time scenarios @ LinkedIn
>> * U-Replicator : Uber's multi-datacenter kafka mirroring service
>> * Concourse : LinkedIn’s near-real-time targeting and scoring platform
>> for notifications built on top of our ML and Stream Processing Infra.
>>
>> You can sign up at https://lnkd.in/gz3WcWb
>>
>> This time we will be open at both our Sunnyvale and San Francisco
>> office. Looking forward to see you!
>>
>> Best,
>>
>> -Yi
>>
>>
>
Re: Announce of the next stream processing meetup @LinkedIn
Posted by Yi Pan <ni...@gmail.com>.
Just a reminder and some more details of the coming meetup, cheers!
Hi Kafka, Brooklin and Samza Users,
The Streams Infra team invites you to attend the Streams Processing meetup
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>
on July 18th 2018. This meetup focuses on Apache Kafka, Apache Samza, and
related streaming technologies. We will host the actual event at LinkedIn
Sunnyvale office, and in addition to that, we will also host a "*viewing
room*" from San Francisco.This time we have Xinyu Liu
<https://www.linkedin.com/in/xinyu-liu-b0b21648/> from the Samza team
talking about Apache Beam <https://beam.apache.org/> runner for Samza
<https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/BEAM>. The Beam
runner provides an ability to write-once but execute the same job in
multiple environments (e.g. Hadoop for Batch Processing or Samza in
Nearline). It also opens up possibilities for supporting different
languages for stream processing (e.g. Python Applications on Samza). Our
second speaker Hongliang Xu <https://www.linkedin.com/in/hongliangxu/> is
from the Infrastructure team @Uber. His team recently built uReplicator to
replicate data across Kafka clusters. You can find a blog
<https://eng.uber.com/ureplicator/> about the original version of
uReplicator here <https://eng.uber.com/ureplicator/> for reference. In his
talk, Hongliang will focus on the new version of uReplicator, its
architecture and share some of their learnings .Ajith Muralidharan
<https://www.linkedin.com/in/ajithmuralidharan/> & Vivek Nelamangala
<https://www.linkedin.com/in/viveknelamangala/> from LinkedIn will talk
about how they built a near real time targeting and scoring platform
(Concourse) for LinkedIn Notifications. Concourse is one of the largest
Samza Jobs at LinkedIn and if you are building large scale streaming
applications, this is the talk to attend.Below are some additional details
about the talks. If you are interested to attend, Please RSVP via meetup.com
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>.
You can also find additional details (streaming link, location, etc.) in
the meetup link
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/251481797/>.
Hope to see you there!
*Location*:
Main Event -
Yosemite Conference Room, LinkedIn Corporate HQ in Sunnyvale.
2nd floor of 605 W Maude Ave, Sunnyvale, CA.
Viewing Party -
Lotta’s Fountain Conference Room, LinkedIn in San Francisco at 222 2nd
Street, San Francisco, CA.
Agenda:6PM: Doors open6-6:35 PM: Networking6:35-7:10 PM: Beam me up Samza:
How we built a Samza Runner for Apache Beam (Speaker: Xinyu Liu, LinkedIn)
Apache Beam provides an easy-to-use, and powerful model for
state-of-the-art stream and batch processing, portability across a variety
of languages, and the ability to converge offline and nearline data
processing. At LinkedIn, we have developed a Samza Runner to leverage the
cutting-edge features of Beam. This runner combines the large-scale
streaming processing capabilities and first-class state support in Samza
with the advancements in Beam data processing. In this talk, we will
discuss the Beam API and its implementation in Samza and the benefits of
Beam Runner to the Samza and Beam community.
7:15-7:50 PM: uReplicator: Uber Engineering’s Scalable Robust Kafka
Replicator(Speaker: Hongliang Xu, Uber)
At Uber, we operate 20+ Kafka clusters to collect system and application
logs as well as event data from rider and driver apps. We need a Kafka
replication solution to replicate data between Kafka clusters across
multiple data centers for different purposes. This talk will introduce the
history behind uReplicator and the high level architecture. As the original
uReplicator ran into scalability challenges and operational overhead as the
scale of Kafka clusters increased, we built the Federated uReplicator which
addressed above issues and provide an extensible architecture for further
scaling.
7:55-8:30 PM: Concourse - Near real time notifications platform at Linkedin
(Speakers: Ajith Muralidharan & Vivek Nelamangala, LinkedIn)
Concourse is LinkedIn’s first near-real-time targeting and scoring platform
for notifications. In this talk, we will provide an in-depth overview of
the design and discuss various scaling optimizations. We'll explain how
Concourse can score millions of notifications per second, while supporting
the use of feature-rich machine learning models based on terabytes of
feature data.
8:30-9PM: Additional networking and Q&AThank you,Streams Infra @ LinkedIn
On Wed, Jun 13, 2018 at 10:56 PM, Yi Pan <ni...@gmail.com> wrote:
> Hi, all,
>
> We have planed for some super-exciting talks at our next Streams Meetup on
> July 19.
>
> * Beam-Samza integration enabling new real time scenarios @ LinkedIn
> * U-Replicator : Uber's multi-datacenter kafka mirroring service
> * Concourse : LinkedIn’s near-real-time targeting and scoring platform
> for notifications built on top of our ML and Stream Processing Infra.
>
> You can sign up at https://lnkd.in/gz3WcWb
>
> This time we will be open at both our Sunnyvale and San Francisco office.
> Looking forward to see you!
>
> Best,
>
> -Yi
>
>