You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Becket Qin <be...@gmail.com> on 2017/11/17 19:33:04 UTC
Stream Processing Meetup@LinkedIn (Dec 4th)
Hi Kafka users and developers,
We are going to host our quarterly Stream Processing Meetup@LinkedIn on Dec
4. There will be three speakers from Slack, Uber and LinkedIn. Please check
the details below if you are interested.
Thanks,
Jiangjie (Becket) Qin
*Stream Processing with Apache Kafka & Apache Samza*
- Meetup Link: here
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/244889719/>
- When: Dec 4th 2017 @ 6:00pm
- Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA (edit
map
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/244889719/>
)
*Abstract*
1. Stream processing using Samza-SQL @ LinkedIn
*Speaker: Srinivasulu Punuru, LinkedIn*
Imagine if you can develop and run a stream processing job in few minutes
and Imagine if a vast majority of your organization (business analysts,
Product manager, Data scientists) can do this on their own without a need
for a development team.
Need for real time insights into the big data is increasing at a rapid
pace. The traditional Java based development model of developing, deploying
and managing the stream processing application is becoming a huge
constraint.
With Samza SQL we can simplify application development by enabling users to
create stream processing applications and get real time insights into their
business using SQL statements.
In this talk we try to answer the following questions
1. How SQL language can be used to perform stream processing?
2. How is Samza SQL implemented - Architecture?
3. How can you deploy Samza SQL in your company?
2. Streaming data pipeline @ Slack
*Speaker:- Ananth Packkildurai, Slack*
*Abstract: *Slack is a communication and collaboration platform for teams.
Our millions of users spend 10+ hrs connected to the service on a typical
working day. They expect reliability, low latency, and extraordinarily rich
client experiences across a wide variety of devices and network conditions.
It is crucial for the developers to get the realtime insights on Slack
operational metrics.
In this talk, I will talk about how our data platform evolves from the
batch system to near realtime. I will also touch base on how Samza helps us
to build low latency data pipelines & Experimentation framework.
3. Improving Kafka at-least-once performance
*Speaker: Ying Zheng, Uber*
*Abstract:*
Abstract:
At Uber, we are seeing an increased demand for Kafka at-least-once
delivery. So far, we are running a dedicated at-least-once Kafka cluster
with special settings. With a very low workload, the dedicated
at-least-once cluster has been working well for more than a year. Now, when
we want to turn on at-least-once producing on all the Kafka clusters, the
at-least-once producing performance is one of the concerns. I have worked a
couple of months to investigate the Kafka performance issues. With Kafka
code changes and Kafka / Java configuration changes, I have reduced
at-least-once producing latency by about 60% to 70%. Some of those
improvements can also improve the general Kafka throughput or reducing
end-to-end Kafka latency, when ack = 0 or ack = 1.
Re: Stream Processing Meetup@LinkedIn (Dec 4th)
Posted by Kailis Dewer <ka...@gmail.com>.
Can some one post the link to the stream, please.
Thanks.
On Fri, Nov 17, 2017 at 3:49 PM, Becket Qin <be...@gmail.com> wrote:
> Hi Paolo,
>
> Yes, we will stream the meetup. Usually the link will be posted to the
> meetup website a couple of hours before the meetup. Feel free to ping me if
> you don't see it.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Fri, Nov 17, 2017 at 11:59 AM, Paolo Patierno <pp...@live.com>
> wrote:
>
> > Hi Becket,
> > I watched some of these meetups on the related YouTube channel in the
> past.
> > Will be it available in streaming or just recorded for watching it later
> ?
> >
> > Thanks
> > Paolo
> > ________________________________
> > From: Becket Qin <be...@gmail.com>
> > Sent: Friday, November 17, 2017 8:33:04 PM
> > To: dev@kafka.apache.org; users@kafka.apache.org
> > Subject: Stream Processing Meetup@LinkedIn (Dec 4th)
> >
> > Hi Kafka users and developers,
> >
> > We are going to host our quarterly Stream Processing Meetup@LinkedIn on
> > Dec
> > 4. There will be three speakers from Slack, Uber and LinkedIn. Please
> check
> > the details below if you are interested.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > *Stream Processing with Apache Kafka & Apache Samza*
> >
> > - Meetup Link: here
> > <https://www.meetup.com/Stream-Processing-Meetup-
> > LinkedIn/events/244889719/>
> > - When: Dec 4th 2017 @ 6:00pm
> > - Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA
> > (edit
> > map
> > <https://www.meetup.com/Stream-Processing-Meetup-
> > LinkedIn/events/244889719/>
> > )
> >
> >
> > *Abstract*
> >
> > 1. Stream processing using Samza-SQL @ LinkedIn
> >
> > *Speaker: Srinivasulu Punuru, LinkedIn*
> > Imagine if you can develop and run a stream processing job in few minutes
> > and Imagine if a vast majority of your organization (business analysts,
> > Product manager, Data scientists) can do this on their own without a need
> > for a development team.
> > Need for real time insights into the big data is increasing at a rapid
> > pace. The traditional Java based development model of developing,
> deploying
> > and managing the stream processing application is becoming a huge
> > constraint.
> > With Samza SQL we can simplify application development by enabling users
> to
> > create stream processing applications and get real time insights into
> their
> > business using SQL statements.
> >
> > In this talk we try to answer the following questions
> >
> > 1. How SQL language can be used to perform stream processing?
> > 2. How is Samza SQL implemented - Architecture?
> > 3. How can you deploy Samza SQL in your company?
> >
> >
> > 2. Streaming data pipeline @ Slack
> > *Speaker:- Ananth Packkildurai, Slack*
> > *Abstract: *Slack is a communication and collaboration platform for
> teams.
> > Our millions of users spend 10+ hrs connected to the service on a typical
> > working day. They expect reliability, low latency, and extraordinarily
> rich
> > client experiences across a wide variety of devices and network
> conditions.
> > It is crucial for the developers to get the realtime insights on Slack
> > operational metrics.
> > In this talk, I will talk about how our data platform evolves from the
> > batch system to near realtime. I will also touch base on how Samza helps
> us
> > to build low latency data pipelines & Experimentation framework.
> >
> > 3. Improving Kafka at-least-once performance
> > *Speaker: Ying Zheng, Uber*
> > *Abstract:*
> > Abstract:
> > At Uber, we are seeing an increased demand for Kafka at-least-once
> > delivery. So far, we are running a dedicated at-least-once Kafka cluster
> > with special settings. With a very low workload, the dedicated
> > at-least-once cluster has been working well for more than a year. Now,
> when
> > we want to turn on at-least-once producing on all the Kafka clusters, the
> > at-least-once producing performance is one of the concerns. I have
> worked a
> > couple of months to investigate the Kafka performance issues. With Kafka
> > code changes and Kafka / Java configuration changes, I have reduced
> > at-least-once producing latency by about 60% to 70%. Some of those
> > improvements can also improve the general Kafka throughput or reducing
> > end-to-end Kafka latency, when ack = 0 or ack = 1.
> >
>
Re: Stream Processing Meetup@LinkedIn (Dec 4th)
Posted by Kailis Dewer <ka...@gmail.com>.
Can some one post the link to the stream, please.
Thanks.
On Fri, Nov 17, 2017 at 3:49 PM, Becket Qin <be...@gmail.com> wrote:
> Hi Paolo,
>
> Yes, we will stream the meetup. Usually the link will be posted to the
> meetup website a couple of hours before the meetup. Feel free to ping me if
> you don't see it.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Fri, Nov 17, 2017 at 11:59 AM, Paolo Patierno <pp...@live.com>
> wrote:
>
> > Hi Becket,
> > I watched some of these meetups on the related YouTube channel in the
> past.
> > Will be it available in streaming or just recorded for watching it later
> ?
> >
> > Thanks
> > Paolo
> > ________________________________
> > From: Becket Qin <be...@gmail.com>
> > Sent: Friday, November 17, 2017 8:33:04 PM
> > To: dev@kafka.apache.org; users@kafka.apache.org
> > Subject: Stream Processing Meetup@LinkedIn (Dec 4th)
> >
> > Hi Kafka users and developers,
> >
> > We are going to host our quarterly Stream Processing Meetup@LinkedIn on
> > Dec
> > 4. There will be three speakers from Slack, Uber and LinkedIn. Please
> check
> > the details below if you are interested.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > *Stream Processing with Apache Kafka & Apache Samza*
> >
> > - Meetup Link: here
> > <https://www.meetup.com/Stream-Processing-Meetup-
> > LinkedIn/events/244889719/>
> > - When: Dec 4th 2017 @ 6:00pm
> > - Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA
> > (edit
> > map
> > <https://www.meetup.com/Stream-Processing-Meetup-
> > LinkedIn/events/244889719/>
> > )
> >
> >
> > *Abstract*
> >
> > 1. Stream processing using Samza-SQL @ LinkedIn
> >
> > *Speaker: Srinivasulu Punuru, LinkedIn*
> > Imagine if you can develop and run a stream processing job in few minutes
> > and Imagine if a vast majority of your organization (business analysts,
> > Product manager, Data scientists) can do this on their own without a need
> > for a development team.
> > Need for real time insights into the big data is increasing at a rapid
> > pace. The traditional Java based development model of developing,
> deploying
> > and managing the stream processing application is becoming a huge
> > constraint.
> > With Samza SQL we can simplify application development by enabling users
> to
> > create stream processing applications and get real time insights into
> their
> > business using SQL statements.
> >
> > In this talk we try to answer the following questions
> >
> > 1. How SQL language can be used to perform stream processing?
> > 2. How is Samza SQL implemented - Architecture?
> > 3. How can you deploy Samza SQL in your company?
> >
> >
> > 2. Streaming data pipeline @ Slack
> > *Speaker:- Ananth Packkildurai, Slack*
> > *Abstract: *Slack is a communication and collaboration platform for
> teams.
> > Our millions of users spend 10+ hrs connected to the service on a typical
> > working day. They expect reliability, low latency, and extraordinarily
> rich
> > client experiences across a wide variety of devices and network
> conditions.
> > It is crucial for the developers to get the realtime insights on Slack
> > operational metrics.
> > In this talk, I will talk about how our data platform evolves from the
> > batch system to near realtime. I will also touch base on how Samza helps
> us
> > to build low latency data pipelines & Experimentation framework.
> >
> > 3. Improving Kafka at-least-once performance
> > *Speaker: Ying Zheng, Uber*
> > *Abstract:*
> > Abstract:
> > At Uber, we are seeing an increased demand for Kafka at-least-once
> > delivery. So far, we are running a dedicated at-least-once Kafka cluster
> > with special settings. With a very low workload, the dedicated
> > at-least-once cluster has been working well for more than a year. Now,
> when
> > we want to turn on at-least-once producing on all the Kafka clusters, the
> > at-least-once producing performance is one of the concerns. I have
> worked a
> > couple of months to investigate the Kafka performance issues. With Kafka
> > code changes and Kafka / Java configuration changes, I have reduced
> > at-least-once producing latency by about 60% to 70%. Some of those
> > improvements can also improve the general Kafka throughput or reducing
> > end-to-end Kafka latency, when ack = 0 or ack = 1.
> >
>
Re: Stream Processing Meetup@LinkedIn (Dec 4th)
Posted by Becket Qin <be...@gmail.com>.
Hi Paolo,
Yes, we will stream the meetup. Usually the link will be posted to the
meetup website a couple of hours before the meetup. Feel free to ping me if
you don't see it.
Thanks,
Jiangjie (Becket) Qin
On Fri, Nov 17, 2017 at 11:59 AM, Paolo Patierno <pp...@live.com> wrote:
> Hi Becket,
> I watched some of these meetups on the related YouTube channel in the past.
> Will be it available in streaming or just recorded for watching it later ?
>
> Thanks
> Paolo
> ________________________________
> From: Becket Qin <be...@gmail.com>
> Sent: Friday, November 17, 2017 8:33:04 PM
> To: dev@kafka.apache.org; users@kafka.apache.org
> Subject: Stream Processing Meetup@LinkedIn (Dec 4th)
>
> Hi Kafka users and developers,
>
> We are going to host our quarterly Stream Processing Meetup@LinkedIn on
> Dec
> 4. There will be three speakers from Slack, Uber and LinkedIn. Please check
> the details below if you are interested.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> *Stream Processing with Apache Kafka & Apache Samza*
>
> - Meetup Link: here
> <https://www.meetup.com/Stream-Processing-Meetup-
> LinkedIn/events/244889719/>
> - When: Dec 4th 2017 @ 6:00pm
> - Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA
> (edit
> map
> <https://www.meetup.com/Stream-Processing-Meetup-
> LinkedIn/events/244889719/>
> )
>
>
> *Abstract*
>
> 1. Stream processing using Samza-SQL @ LinkedIn
>
> *Speaker: Srinivasulu Punuru, LinkedIn*
> Imagine if you can develop and run a stream processing job in few minutes
> and Imagine if a vast majority of your organization (business analysts,
> Product manager, Data scientists) can do this on their own without a need
> for a development team.
> Need for real time insights into the big data is increasing at a rapid
> pace. The traditional Java based development model of developing, deploying
> and managing the stream processing application is becoming a huge
> constraint.
> With Samza SQL we can simplify application development by enabling users to
> create stream processing applications and get real time insights into their
> business using SQL statements.
>
> In this talk we try to answer the following questions
>
> 1. How SQL language can be used to perform stream processing?
> 2. How is Samza SQL implemented - Architecture?
> 3. How can you deploy Samza SQL in your company?
>
>
> 2. Streaming data pipeline @ Slack
> *Speaker:- Ananth Packkildurai, Slack*
> *Abstract: *Slack is a communication and collaboration platform for teams.
> Our millions of users spend 10+ hrs connected to the service on a typical
> working day. They expect reliability, low latency, and extraordinarily rich
> client experiences across a wide variety of devices and network conditions.
> It is crucial for the developers to get the realtime insights on Slack
> operational metrics.
> In this talk, I will talk about how our data platform evolves from the
> batch system to near realtime. I will also touch base on how Samza helps us
> to build low latency data pipelines & Experimentation framework.
>
> 3. Improving Kafka at-least-once performance
> *Speaker: Ying Zheng, Uber*
> *Abstract:*
> Abstract:
> At Uber, we are seeing an increased demand for Kafka at-least-once
> delivery. So far, we are running a dedicated at-least-once Kafka cluster
> with special settings. With a very low workload, the dedicated
> at-least-once cluster has been working well for more than a year. Now, when
> we want to turn on at-least-once producing on all the Kafka clusters, the
> at-least-once producing performance is one of the concerns. I have worked a
> couple of months to investigate the Kafka performance issues. With Kafka
> code changes and Kafka / Java configuration changes, I have reduced
> at-least-once producing latency by about 60% to 70%. Some of those
> improvements can also improve the general Kafka throughput or reducing
> end-to-end Kafka latency, when ack = 0 or ack = 1.
>
Re: Stream Processing Meetup@LinkedIn (Dec 4th)
Posted by Becket Qin <be...@gmail.com>.
Hi Paolo,
Yes, we will stream the meetup. Usually the link will be posted to the
meetup website a couple of hours before the meetup. Feel free to ping me if
you don't see it.
Thanks,
Jiangjie (Becket) Qin
On Fri, Nov 17, 2017 at 11:59 AM, Paolo Patierno <pp...@live.com> wrote:
> Hi Becket,
> I watched some of these meetups on the related YouTube channel in the past.
> Will be it available in streaming or just recorded for watching it later ?
>
> Thanks
> Paolo
> ________________________________
> From: Becket Qin <be...@gmail.com>
> Sent: Friday, November 17, 2017 8:33:04 PM
> To: dev@kafka.apache.org; users@kafka.apache.org
> Subject: Stream Processing Meetup@LinkedIn (Dec 4th)
>
> Hi Kafka users and developers,
>
> We are going to host our quarterly Stream Processing Meetup@LinkedIn on
> Dec
> 4. There will be three speakers from Slack, Uber and LinkedIn. Please check
> the details below if you are interested.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> *Stream Processing with Apache Kafka & Apache Samza*
>
> - Meetup Link: here
> <https://www.meetup.com/Stream-Processing-Meetup-
> LinkedIn/events/244889719/>
> - When: Dec 4th 2017 @ 6:00pm
> - Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA
> (edit
> map
> <https://www.meetup.com/Stream-Processing-Meetup-
> LinkedIn/events/244889719/>
> )
>
>
> *Abstract*
>
> 1. Stream processing using Samza-SQL @ LinkedIn
>
> *Speaker: Srinivasulu Punuru, LinkedIn*
> Imagine if you can develop and run a stream processing job in few minutes
> and Imagine if a vast majority of your organization (business analysts,
> Product manager, Data scientists) can do this on their own without a need
> for a development team.
> Need for real time insights into the big data is increasing at a rapid
> pace. The traditional Java based development model of developing, deploying
> and managing the stream processing application is becoming a huge
> constraint.
> With Samza SQL we can simplify application development by enabling users to
> create stream processing applications and get real time insights into their
> business using SQL statements.
>
> In this talk we try to answer the following questions
>
> 1. How SQL language can be used to perform stream processing?
> 2. How is Samza SQL implemented - Architecture?
> 3. How can you deploy Samza SQL in your company?
>
>
> 2. Streaming data pipeline @ Slack
> *Speaker:- Ananth Packkildurai, Slack*
> *Abstract: *Slack is a communication and collaboration platform for teams.
> Our millions of users spend 10+ hrs connected to the service on a typical
> working day. They expect reliability, low latency, and extraordinarily rich
> client experiences across a wide variety of devices and network conditions.
> It is crucial for the developers to get the realtime insights on Slack
> operational metrics.
> In this talk, I will talk about how our data platform evolves from the
> batch system to near realtime. I will also touch base on how Samza helps us
> to build low latency data pipelines & Experimentation framework.
>
> 3. Improving Kafka at-least-once performance
> *Speaker: Ying Zheng, Uber*
> *Abstract:*
> Abstract:
> At Uber, we are seeing an increased demand for Kafka at-least-once
> delivery. So far, we are running a dedicated at-least-once Kafka cluster
> with special settings. With a very low workload, the dedicated
> at-least-once cluster has been working well for more than a year. Now, when
> we want to turn on at-least-once producing on all the Kafka clusters, the
> at-least-once producing performance is one of the concerns. I have worked a
> couple of months to investigate the Kafka performance issues. With Kafka
> code changes and Kafka / Java configuration changes, I have reduced
> at-least-once producing latency by about 60% to 70%. Some of those
> improvements can also improve the general Kafka throughput or reducing
> end-to-end Kafka latency, when ack = 0 or ack = 1.
>
Re: Stream Processing Meetup@LinkedIn (Dec 4th)
Posted by Paolo Patierno <pp...@live.com>.
Hi Becket,
I watched some of these meetups on the related YouTube channel in the past.
Will be it available in streaming or just recorded for watching it later ?
Thanks
Paolo
________________________________
From: Becket Qin <be...@gmail.com>
Sent: Friday, November 17, 2017 8:33:04 PM
To: dev@kafka.apache.org; users@kafka.apache.org
Subject: Stream Processing Meetup@LinkedIn (Dec 4th)
Hi Kafka users and developers,
We are going to host our quarterly Stream Processing Meetup@LinkedIn on Dec
4. There will be three speakers from Slack, Uber and LinkedIn. Please check
the details below if you are interested.
Thanks,
Jiangjie (Becket) Qin
*Stream Processing with Apache Kafka & Apache Samza*
- Meetup Link: here
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/244889719/>
- When: Dec 4th 2017 @ 6:00pm
- Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA (edit
map
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/244889719/>
)
*Abstract*
1. Stream processing using Samza-SQL @ LinkedIn
*Speaker: Srinivasulu Punuru, LinkedIn*
Imagine if you can develop and run a stream processing job in few minutes
and Imagine if a vast majority of your organization (business analysts,
Product manager, Data scientists) can do this on their own without a need
for a development team.
Need for real time insights into the big data is increasing at a rapid
pace. The traditional Java based development model of developing, deploying
and managing the stream processing application is becoming a huge
constraint.
With Samza SQL we can simplify application development by enabling users to
create stream processing applications and get real time insights into their
business using SQL statements.
In this talk we try to answer the following questions
1. How SQL language can be used to perform stream processing?
2. How is Samza SQL implemented - Architecture?
3. How can you deploy Samza SQL in your company?
2. Streaming data pipeline @ Slack
*Speaker:- Ananth Packkildurai, Slack*
*Abstract: *Slack is a communication and collaboration platform for teams.
Our millions of users spend 10+ hrs connected to the service on a typical
working day. They expect reliability, low latency, and extraordinarily rich
client experiences across a wide variety of devices and network conditions.
It is crucial for the developers to get the realtime insights on Slack
operational metrics.
In this talk, I will talk about how our data platform evolves from the
batch system to near realtime. I will also touch base on how Samza helps us
to build low latency data pipelines & Experimentation framework.
3. Improving Kafka at-least-once performance
*Speaker: Ying Zheng, Uber*
*Abstract:*
Abstract:
At Uber, we are seeing an increased demand for Kafka at-least-once
delivery. So far, we are running a dedicated at-least-once Kafka cluster
with special settings. With a very low workload, the dedicated
at-least-once cluster has been working well for more than a year. Now, when
we want to turn on at-least-once producing on all the Kafka clusters, the
at-least-once producing performance is one of the concerns. I have worked a
couple of months to investigate the Kafka performance issues. With Kafka
code changes and Kafka / Java configuration changes, I have reduced
at-least-once producing latency by about 60% to 70%. Some of those
improvements can also improve the general Kafka throughput or reducing
end-to-end Kafka latency, when ack = 0 or ack = 1.
Re: Stream Processing Meetup@LinkedIn (Dec 4th)
Posted by Paolo Patierno <pp...@live.com>.
Hi Becket,
I watched some of these meetups on the related YouTube channel in the past.
Will be it available in streaming or just recorded for watching it later ?
Thanks
Paolo
________________________________
From: Becket Qin <be...@gmail.com>
Sent: Friday, November 17, 2017 8:33:04 PM
To: dev@kafka.apache.org; users@kafka.apache.org
Subject: Stream Processing Meetup@LinkedIn (Dec 4th)
Hi Kafka users and developers,
We are going to host our quarterly Stream Processing Meetup@LinkedIn on Dec
4. There will be three speakers from Slack, Uber and LinkedIn. Please check
the details below if you are interested.
Thanks,
Jiangjie (Becket) Qin
*Stream Processing with Apache Kafka & Apache Samza*
- Meetup Link: here
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/244889719/>
- When: Dec 4th 2017 @ 6:00pm
- Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA (edit
map
<https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/244889719/>
)
*Abstract*
1. Stream processing using Samza-SQL @ LinkedIn
*Speaker: Srinivasulu Punuru, LinkedIn*
Imagine if you can develop and run a stream processing job in few minutes
and Imagine if a vast majority of your organization (business analysts,
Product manager, Data scientists) can do this on their own without a need
for a development team.
Need for real time insights into the big data is increasing at a rapid
pace. The traditional Java based development model of developing, deploying
and managing the stream processing application is becoming a huge
constraint.
With Samza SQL we can simplify application development by enabling users to
create stream processing applications and get real time insights into their
business using SQL statements.
In this talk we try to answer the following questions
1. How SQL language can be used to perform stream processing?
2. How is Samza SQL implemented - Architecture?
3. How can you deploy Samza SQL in your company?
2. Streaming data pipeline @ Slack
*Speaker:- Ananth Packkildurai, Slack*
*Abstract: *Slack is a communication and collaboration platform for teams.
Our millions of users spend 10+ hrs connected to the service on a typical
working day. They expect reliability, low latency, and extraordinarily rich
client experiences across a wide variety of devices and network conditions.
It is crucial for the developers to get the realtime insights on Slack
operational metrics.
In this talk, I will talk about how our data platform evolves from the
batch system to near realtime. I will also touch base on how Samza helps us
to build low latency data pipelines & Experimentation framework.
3. Improving Kafka at-least-once performance
*Speaker: Ying Zheng, Uber*
*Abstract:*
Abstract:
At Uber, we are seeing an increased demand for Kafka at-least-once
delivery. So far, we are running a dedicated at-least-once Kafka cluster
with special settings. With a very low workload, the dedicated
at-least-once cluster has been working well for more than a year. Now, when
we want to turn on at-least-once producing on all the Kafka clusters, the
at-least-once producing performance is one of the concerns. I have worked a
couple of months to investigate the Kafka performance issues. With Kafka
code changes and Kafka / Java configuration changes, I have reduced
at-least-once producing latency by about 60% to 70%. Some of those
improvements can also improve the general Kafka throughput or reducing
end-to-end Kafka latency, when ack = 0 or ack = 1.