You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Taylor Gautier <tg...@tagged.com> on 2011/11/18 18:07:33 UTC
We're live!
I just wanted to let you guys know we are live with our first Kafka use case.
You'll see more from me in the coming weeks about it.
We released the feature for the public on Monday of this week and have
been slowly rolling it out to our users.
We will continue to gradually ramp up usage until we get to 100%.
There is still a lot to do with our implementation and more use cases
are coming soon.
But so far I am very happy with Kafka it has met all of my
expectations and lived up to all of the claims made by the LinkedIn
team.
Thank you guys so much for OSS'ing such a great piece of technology.
Re: We're live!
Posted by Jun Rao <ju...@gmail.com>.
Taylor,
Thanks for sharing this. Do you think that you can add your usage to our
wiki?
Look forward to various contributions (bug reports, patches, etc) from you
to make this project better.
Jun
On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <tg...@tagged.com> wrote:
> I just wanted to let you guys know we are live with our first Kafka use
> case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.
>
Re: We're live!
Posted by Evan Chan <ev...@ooyala.com>.
Taylor,
Just out of curiosity, do you guys have to deal with at-least-once message
processing? If so, I'd be curious how you handle all the corner cases
with Kafka, such as what if something downstream fails and you need to
replay some logs.
thanks,
Evan
On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <tg...@tagged.com> wrote:
> I just wanted to let you guys know we are live with our first Kafka use
> case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.
>
--
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>
Re: We're live!
Posted by Taylor Gautier <tg...@tagged.com>.
Our second use case is live. Traffic will be ramping soon. First use case
is continuing to roll out to more of our user base.
On Fri, Nov 18, 2011 at 3:04 PM, Mark <st...@gmail.com> wrote:
> I see that you removed the "Application and Usage" page. I was hoping we
> would get some users of Kafka detailing their use cases rather than the
> default high level "It powers my ..."
>
>
> On 11/18/11 1:37 PM, Olivier Pomel wrote:
>
>> Great! I should say that we're now live with Kafka too - and we'll
>> soon have all our incoming data flow through it.
>>
>> We love it so far, and it's absolutely perfect for our use cases. Great
>> work!
>> Best,
>> Olivier.
>> --
>> Check us out at http://datadoghq.com
>>
>>
>>
>>
>> On Fri, Nov 18, 2011 at 4:31 PM, Neha Narkhede<ne...@gmail.com>>
>> wrote:
>>
>>> Guys,
>>>
>>> I've updated our "Powered By" wiki to include LinkedIn -
>>> https://cwiki.apache.org/**confluence/display/KAFKA/**Powered+By<https://cwiki.apache.org/confluence/display/KAFKA/Powered+By>
>>>
>>> Please feel free to update it to include your respective companies and a
>>> short description of the usage.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao<ju...@gmail.com> wrote:
>>>
>>> Dave,
>>>>
>>>> Thanks for the update. Could you add your use case to our wiki too?
>>>>
>>>> Look forward to feedback and contributions from you in the future.
>>>>
>>>> Jun
>>>>
>>>>
Re: We're live!
Posted by Mark <st...@gmail.com>.
I see that you removed the "Application and Usage" page. I was hoping we
would get some users of Kafka detailing their use cases rather than the
default high level "It powers my ..."
On 11/18/11 1:37 PM, Olivier Pomel wrote:
> Great! I should say that we're now live with Kafka too - and we'll
> soon have all our incoming data flow through it.
>
> We love it so far, and it's absolutely perfect for our use cases. Great work!
> Best,
> Olivier.
> --
> Check us out at http://datadoghq.com
>
>
>
>
> On Fri, Nov 18, 2011 at 4:31 PM, Neha Narkhede<ne...@gmail.com> wrote:
>> Guys,
>>
>> I've updated our "Powered By" wiki to include LinkedIn -
>> https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
>>
>> Please feel free to update it to include your respective companies and a
>> short description of the usage.
>>
>> Thanks,
>> Neha
>>
>> On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao<ju...@gmail.com> wrote:
>>
>>> Dave,
>>>
>>> Thanks for the update. Could you add your use case to our wiki too?
>>>
>>> Look forward to feedback and contributions from you in the future.
>>>
>>> Jun
>>>
Re: We're live!
Posted by Olivier Pomel <ol...@datadoghq.com>.
Great! I should say that we're now live with Kafka too - and we'll
soon have all our incoming data flow through it.
We love it so far, and it's absolutely perfect for our use cases. Great work!
Best,
Olivier.
--
Check us out at http://datadoghq.com
On Fri, Nov 18, 2011 at 4:31 PM, Neha Narkhede <ne...@gmail.com> wrote:
> Guys,
>
> I've updated our "Powered By" wiki to include LinkedIn -
> https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
>
> Please feel free to update it to include your respective companies and a
> short description of the usage.
>
> Thanks,
> Neha
>
> On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Dave,
>>
>> Thanks for the update. Could you add your use case to our wiki too?
>>
>> Look forward to feedback and contributions from you in the future.
>>
>> Jun
>>
>
Re: We're live!
Posted by Neha Narkhede <ne...@gmail.com>.
Guys,
I've updated our "Powered By" wiki to include LinkedIn -
https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
Please feel free to update it to include your respective companies and a
short description of the usage.
Thanks,
Neha
On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao <ju...@gmail.com> wrote:
> Dave,
>
> Thanks for the update. Could you add your use case to our wiki too?
>
> Look forward to feedback and contributions from you in the future.
>
> Jun
>
Re: We're live!
Posted by Jun Rao <ju...@gmail.com>.
Dave,
Thanks for the update. Could you add your use case to our wiki too?
Look forward to feedback and contributions from you in the future.
Jun
Re: We're live!
Posted by Dave Fayram <df...@gmail.com>.
Not to be left out, our first Kafka use case at Crowdflower went live
on Monday. We've completely replaced a rather large portion of
postgres-sweep-based statistics calculations with 2 kafka data
channels (handling a major fraction of all the data we touch) and a
handful of scala processes. Working well so far, and more uses cases
from this data will result over time.
The net result is a rather dramatic drop in write and read load to our
master postgres cluster. Our engineering blog will probably talk about
it publicly soon, because we're really quite happy with the resulting
system; it's simple, performs well, and is quite extensible thanks to
the low overhead of adding new consumer groups to existing channels.
- dlf
On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <tg...@tagged.com> wrote:
> I just wanted to let you guys know we are live with our first Kafka use case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.
>
--
--
Dave Fayram
dfayram@gmail.com
Re: We're live!
Posted by Chris Burroughs <ch...@gmail.com>.
Congratulations! Looking forward to reading more about what your
implementation.
On 11/18/2011 12:07 PM, Taylor Gautier wrote:
> I just wanted to let you guys know we are live with our first Kafka use case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.