You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Nitin Lamba <nl...@apache.org> on 2017/02/20 07:47:32 UTC
RE: Beam on Kubernetes
Hi,
Trying to restart this thread from last November [1]. Packaging an
end-2-end Beam example for k8s environment, similar to the one from the
TensorFlow team [2], may be interesting to look at. The logical progression
is:
- Start with an example using the local (java) runner
- Build the next one for Spark; k8s repo already has Spark v1.5.x as an
example [3] that can be updated/ modified
- Other runners to follow using Spark as a template
Let me know if there is interest in pursuing/ collaborating on this.
Thanks,
Nitin
[1]
https://www.mail-archive.com/user@beam.incubator.apache.org/msg00881.html
[2] https://tensorflow.github.io/serving/serving_inception
[3] https://github.com/kubernetes/kubernetes/tree/master/examples/spark
Re: Beam on Kubernetes
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
I can invite you on the slack channel.
Can you send (in private message) the e-mail address you want to use ?
Thanks !
Regards
JB
On 02/22/2017 09:56 AM, Ekrem Aksoy wrote:
> Hi, I think I'm unable to login to this Slack channel it says (I'm just
> using my gmail dot com account or I have a company email but neither works):
>
> If you have an *@apache.org <http://apache.org> or @google.com
> <http://google.com> *email address, you can create an account
> <https://apachebeam.slack.com/signup/>.
>
> On Tue, Feb 21, 2017 at 3:01 PM, Jean-Baptiste Onofr� <jb...@nanthrax.net>
> wrote:
>
>> Hi,
>>
>> Slack channel is on: https://apachebeam.slack.com
>>
>> I'm also happy to help, don't hesitate to ping me on Slack.
>>
>> Regards
>> JB
>>
>>
>> On 02/21/2017 11:23 AM, Ekrem Aksoy wrote:
>>
>>> Hey @Nitin,
>>>
>>> We can talk on Slack (I don't know how to get into Slack channel, btw).
>>> I've local vagrant k8s (for different dev purpose, but I can fire a new
>>> cluster based on that). I'm using AWS stack (ECR/ECS for containers and
>>> CloudFormation things)
>>>
>>> Best,
>>>
>>> Ekrem
>>>
>>> On Tue, Feb 21, 2017 at 11:39 AM, Nitin Lamba <nl...@apache.org> wrote:
>>>
>>> @Ekrem
>>>> Sure! Would be great to work together on this.
>>>>
>>>> I'm assuming you already have the basic docker setup on your dev machine.
>>>> If you're new to kubernetes, the first step is to get a local k8s setup
>>>> going - single node (minikube) or multi-node (vagrant). Alternatively,
>>>> you
>>>> can setup a container environment on GCP or AWS. It'll help packaging and
>>>> test-driving the images/ pods as they're built.
>>>>
>>>> Let me know if you'd like to chat real time (on Slack) about it.
>>>>
>>>> Thanks,
>>>> Nitin
>>>> On Mon, Feb 20, 2017 at 9:41 PM Ekrem Aksoy <ek...@gmail.com>
>>>> wrote:
>>>>
>>>> I'm also interested in this effort. How can I contribute?
>>>>>
>>>>>
>>>>> Best,
>>>>>
>>>>> Ekrem
>>>>>
>>>>> On Tue, Feb 21, 2017 at 4:12 AM, Nitin Lamba <ni...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Thanks JB, Davor!
>>>>>>
>>>>>> Yes, the step-by-step Spark/Mesos guide will certainly help.
>>>>>>
>>>>>> I'll start with packaging java runner/ examples first and see how far I
>>>>>> get. Will create a JIRA and should have an update in the next week or
>>>>>>
>>>>> so.
>>>>
>>>>>
>>>>>> Best,
>>>>>> Nitin
>>>>>>
>>>>>> On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org>
>>>>>>
>>>>> wrote:
>>>>
>>>>>
>>>>>> I think these are great ideas for simplifying the getting started
>>>>>>> experience across runners -- we'd love a contribution in this space!
>>>>>>>
>>>>>>> On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofr� <
>>>>>>>
>>>>>> jb@nanthrax.net>
>>>>>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Nitin,
>>>>>>>>
>>>>>>>> It sounds like a good candidate for blog or documentation, or even
>>>>>>>>
>>>>>>> an
>>>>
>>>>> example.
>>>>>>>>
>>>>>>>> I have a step-by-step example of a pipeline running on Spark with
>>>>>>>>
>>>>>>> Mesos
>>>>>
>>>>>> (not yet Kubernetes) if you are interested.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> JB
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/20/2017 08:47 AM, Nitin Lamba wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Trying to restart this thread from last November [1]. Packaging an
>>>>>>>>> end-2-end Beam example for k8s environment, similar to the one
>>>>>>>>>
>>>>>>>> from
>>>>
>>>>> the
>>>>>>
>>>>>>> TensorFlow team [2], may be interesting to look at. The logical
>>>>>>>>> progression
>>>>>>>>> is:
>>>>>>>>>
>>>>>>>>> - Start with an example using the local (java) runner
>>>>>>>>> - Build the next one for Spark; k8s repo already has Spark v1.5.x
>>>>>>>>>
>>>>>>>> as
>>>>
>>>>> an
>>>>>>
>>>>>>> example [3] that can be updated/ modified
>>>>>>>>> - Other runners to follow using Spark as a template
>>>>>>>>>
>>>>>>>>> Let me know if there is interest in pursuing/ collaborating on
>>>>>>>>>
>>>>>>>> this.
>>>>
>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Nitin
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://www.mail-archive.com/user@beam.incubator.apache.
>>>>>>>>>
>>>>>>>> org/msg00881.html
>>>>>>>
>>>>>>>> [2] https://tensorflow.github.io/serving/serving_inception
>>>>>>>>> [3] https://github.com/kubernetes/kubernetes/tree/master/
>>>>>>>>>
>>>>>>>> examples/spark
>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>> Jean-Baptiste Onofr�
>>>>>>>> jbonofre@apache.org
>>>>>>>> http://blog.nanthrax.net
>>>>>>>> Talend - http://www.talend.com
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> --
>> Jean-Baptiste Onofr�
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
--
Jean-Baptiste Onofr�
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Beam on Kubernetes
Posted by Ekrem Aksoy <ek...@gmail.com>.
Hi, I think I'm unable to login to this Slack channel it says (I'm just
using my gmail dot com account or I have a company email but neither works):
If you have an *@apache.org <http://apache.org> or @google.com
<http://google.com> *email address, you can create an account
<https://apachebeam.slack.com/signup/>.
On Tue, Feb 21, 2017 at 3:01 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi,
>
> Slack channel is on: https://apachebeam.slack.com
>
> I'm also happy to help, don't hesitate to ping me on Slack.
>
> Regards
> JB
>
>
> On 02/21/2017 11:23 AM, Ekrem Aksoy wrote:
>
>> Hey @Nitin,
>>
>> We can talk on Slack (I don't know how to get into Slack channel, btw).
>> I've local vagrant k8s (for different dev purpose, but I can fire a new
>> cluster based on that). I'm using AWS stack (ECR/ECS for containers and
>> CloudFormation things)
>>
>> Best,
>>
>> Ekrem
>>
>> On Tue, Feb 21, 2017 at 11:39 AM, Nitin Lamba <nl...@apache.org> wrote:
>>
>> @Ekrem
>>> Sure! Would be great to work together on this.
>>>
>>> I'm assuming you already have the basic docker setup on your dev machine.
>>> If you're new to kubernetes, the first step is to get a local k8s setup
>>> going - single node (minikube) or multi-node (vagrant). Alternatively,
>>> you
>>> can setup a container environment on GCP or AWS. It'll help packaging and
>>> test-driving the images/ pods as they're built.
>>>
>>> Let me know if you'd like to chat real time (on Slack) about it.
>>>
>>> Thanks,
>>> Nitin
>>> On Mon, Feb 20, 2017 at 9:41 PM Ekrem Aksoy <ek...@gmail.com>
>>> wrote:
>>>
>>> I'm also interested in this effort. How can I contribute?
>>>>
>>>>
>>>> Best,
>>>>
>>>> Ekrem
>>>>
>>>> On Tue, Feb 21, 2017 at 4:12 AM, Nitin Lamba <ni...@gmail.com>
>>>> wrote:
>>>>
>>>> Thanks JB, Davor!
>>>>>
>>>>> Yes, the step-by-step Spark/Mesos guide will certainly help.
>>>>>
>>>>> I'll start with packaging java runner/ examples first and see how far I
>>>>> get. Will create a JIRA and should have an update in the next week or
>>>>>
>>>> so.
>>>
>>>>
>>>>> Best,
>>>>> Nitin
>>>>>
>>>>> On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org>
>>>>>
>>>> wrote:
>>>
>>>>
>>>>> I think these are great ideas for simplifying the getting started
>>>>>> experience across runners -- we'd love a contribution in this space!
>>>>>>
>>>>>> On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofré <
>>>>>>
>>>>> jb@nanthrax.net>
>>>>
>>>>> wrote:
>>>>>>
>>>>>> Hi Nitin,
>>>>>>>
>>>>>>> It sounds like a good candidate for blog or documentation, or even
>>>>>>>
>>>>>> an
>>>
>>>> example.
>>>>>>>
>>>>>>> I have a step-by-step example of a pipeline running on Spark with
>>>>>>>
>>>>>> Mesos
>>>>
>>>>> (not yet Kubernetes) if you are interested.
>>>>>>>
>>>>>>> Regards
>>>>>>> JB
>>>>>>>
>>>>>>>
>>>>>>> On 02/20/2017 08:47 AM, Nitin Lamba wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Trying to restart this thread from last November [1]. Packaging an
>>>>>>>> end-2-end Beam example for k8s environment, similar to the one
>>>>>>>>
>>>>>>> from
>>>
>>>> the
>>>>>
>>>>>> TensorFlow team [2], may be interesting to look at. The logical
>>>>>>>> progression
>>>>>>>> is:
>>>>>>>>
>>>>>>>> - Start with an example using the local (java) runner
>>>>>>>> - Build the next one for Spark; k8s repo already has Spark v1.5.x
>>>>>>>>
>>>>>>> as
>>>
>>>> an
>>>>>
>>>>>> example [3] that can be updated/ modified
>>>>>>>> - Other runners to follow using Spark as a template
>>>>>>>>
>>>>>>>> Let me know if there is interest in pursuing/ collaborating on
>>>>>>>>
>>>>>>> this.
>>>
>>>>
>>>>>>>> Thanks,
>>>>>>>> Nitin
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://www.mail-archive.com/user@beam.incubator.apache.
>>>>>>>>
>>>>>>> org/msg00881.html
>>>>>>
>>>>>>> [2] https://tensorflow.github.io/serving/serving_inception
>>>>>>>> [3] https://github.com/kubernetes/kubernetes/tree/master/
>>>>>>>>
>>>>>>> examples/spark
>>>>>
>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> Jean-Baptiste Onofré
>>>>>>> jbonofre@apache.org
>>>>>>> http://blog.nanthrax.net
>>>>>>> Talend - http://www.talend.com
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Beam on Kubernetes
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
Slack channel is on: https://apachebeam.slack.com
I'm also happy to help, don't hesitate to ping me on Slack.
Regards
JB
On 02/21/2017 11:23 AM, Ekrem Aksoy wrote:
> Hey @Nitin,
>
> We can talk on Slack (I don't know how to get into Slack channel, btw).
> I've local vagrant k8s (for different dev purpose, but I can fire a new
> cluster based on that). I'm using AWS stack (ECR/ECS for containers and
> CloudFormation things)
>
> Best,
>
> Ekrem
>
> On Tue, Feb 21, 2017 at 11:39 AM, Nitin Lamba <nl...@apache.org> wrote:
>
>> @Ekrem
>> Sure! Would be great to work together on this.
>>
>> I'm assuming you already have the basic docker setup on your dev machine.
>> If you're new to kubernetes, the first step is to get a local k8s setup
>> going - single node (minikube) or multi-node (vagrant). Alternatively, you
>> can setup a container environment on GCP or AWS. It'll help packaging and
>> test-driving the images/ pods as they're built.
>>
>> Let me know if you'd like to chat real time (on Slack) about it.
>>
>> Thanks,
>> Nitin
>> On Mon, Feb 20, 2017 at 9:41 PM Ekrem Aksoy <ek...@gmail.com> wrote:
>>
>>> I'm also interested in this effort. How can I contribute?
>>>
>>>
>>> Best,
>>>
>>> Ekrem
>>>
>>> On Tue, Feb 21, 2017 at 4:12 AM, Nitin Lamba <ni...@gmail.com>
>>> wrote:
>>>
>>>> Thanks JB, Davor!
>>>>
>>>> Yes, the step-by-step Spark/Mesos guide will certainly help.
>>>>
>>>> I'll start with packaging java runner/ examples first and see how far I
>>>> get. Will create a JIRA and should have an update in the next week or
>> so.
>>>>
>>>> Best,
>>>> Nitin
>>>>
>>>> On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org>
>> wrote:
>>>>
>>>>> I think these are great ideas for simplifying the getting started
>>>>> experience across runners -- we'd love a contribution in this space!
>>>>>
>>>>> On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofr� <
>>> jb@nanthrax.net>
>>>>> wrote:
>>>>>
>>>>>> Hi Nitin,
>>>>>>
>>>>>> It sounds like a good candidate for blog or documentation, or even
>> an
>>>>>> example.
>>>>>>
>>>>>> I have a step-by-step example of a pipeline running on Spark with
>>> Mesos
>>>>>> (not yet Kubernetes) if you are interested.
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>>
>>>>>> On 02/20/2017 08:47 AM, Nitin Lamba wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Trying to restart this thread from last November [1]. Packaging an
>>>>>>> end-2-end Beam example for k8s environment, similar to the one
>> from
>>>> the
>>>>>>> TensorFlow team [2], may be interesting to look at. The logical
>>>>>>> progression
>>>>>>> is:
>>>>>>>
>>>>>>> - Start with an example using the local (java) runner
>>>>>>> - Build the next one for Spark; k8s repo already has Spark v1.5.x
>> as
>>>> an
>>>>>>> example [3] that can be updated/ modified
>>>>>>> - Other runners to follow using Spark as a template
>>>>>>>
>>>>>>> Let me know if there is interest in pursuing/ collaborating on
>> this.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Nitin
>>>>>>>
>>>>>>> [1]
>>>>>>> https://www.mail-archive.com/user@beam.incubator.apache.
>>>>> org/msg00881.html
>>>>>>> [2] https://tensorflow.github.io/serving/serving_inception
>>>>>>> [3] https://github.com/kubernetes/kubernetes/tree/master/
>>>> examples/spark
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Jean-Baptiste Onofr�
>>>>>> jbonofre@apache.org
>>>>>> http://blog.nanthrax.net
>>>>>> Talend - http://www.talend.com
>>>>>>
>>>>>
>>>>
>>>
>>
>
--
Jean-Baptiste Onofr�
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Beam on Kubernetes
Posted by Ekrem Aksoy <ek...@gmail.com>.
Hey @Nitin,
We can talk on Slack (I don't know how to get into Slack channel, btw).
I've local vagrant k8s (for different dev purpose, but I can fire a new
cluster based on that). I'm using AWS stack (ECR/ECS for containers and
CloudFormation things)
Best,
Ekrem
On Tue, Feb 21, 2017 at 11:39 AM, Nitin Lamba <nl...@apache.org> wrote:
> @Ekrem
> Sure! Would be great to work together on this.
>
> I'm assuming you already have the basic docker setup on your dev machine.
> If you're new to kubernetes, the first step is to get a local k8s setup
> going - single node (minikube) or multi-node (vagrant). Alternatively, you
> can setup a container environment on GCP or AWS. It'll help packaging and
> test-driving the images/ pods as they're built.
>
> Let me know if you'd like to chat real time (on Slack) about it.
>
> Thanks,
> Nitin
> On Mon, Feb 20, 2017 at 9:41 PM Ekrem Aksoy <ek...@gmail.com> wrote:
>
> > I'm also interested in this effort. How can I contribute?
> >
> >
> > Best,
> >
> > Ekrem
> >
> > On Tue, Feb 21, 2017 at 4:12 AM, Nitin Lamba <ni...@gmail.com>
> > wrote:
> >
> > > Thanks JB, Davor!
> > >
> > > Yes, the step-by-step Spark/Mesos guide will certainly help.
> > >
> > > I'll start with packaging java runner/ examples first and see how far I
> > > get. Will create a JIRA and should have an update in the next week or
> so.
> > >
> > > Best,
> > > Nitin
> > >
> > > On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org>
> wrote:
> > >
> > > > I think these are great ideas for simplifying the getting started
> > > > experience across runners -- we'd love a contribution in this space!
> > > >
> > > > On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofré <
> > jb@nanthrax.net>
> > > > wrote:
> > > >
> > > > > Hi Nitin,
> > > > >
> > > > > It sounds like a good candidate for blog or documentation, or even
> an
> > > > > example.
> > > > >
> > > > > I have a step-by-step example of a pipeline running on Spark with
> > Mesos
> > > > > (not yet Kubernetes) if you are interested.
> > > > >
> > > > > Regards
> > > > > JB
> > > > >
> > > > >
> > > > > On 02/20/2017 08:47 AM, Nitin Lamba wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> Trying to restart this thread from last November [1]. Packaging an
> > > > >> end-2-end Beam example for k8s environment, similar to the one
> from
> > > the
> > > > >> TensorFlow team [2], may be interesting to look at. The logical
> > > > >> progression
> > > > >> is:
> > > > >>
> > > > >> - Start with an example using the local (java) runner
> > > > >> - Build the next one for Spark; k8s repo already has Spark v1.5.x
> as
> > > an
> > > > >> example [3] that can be updated/ modified
> > > > >> - Other runners to follow using Spark as a template
> > > > >>
> > > > >> Let me know if there is interest in pursuing/ collaborating on
> this.
> > > > >>
> > > > >> Thanks,
> > > > >> Nitin
> > > > >>
> > > > >> [1]
> > > > >> https://www.mail-archive.com/user@beam.incubator.apache.
> > > > org/msg00881.html
> > > > >> [2] https://tensorflow.github.io/serving/serving_inception
> > > > >> [3] https://github.com/kubernetes/kubernetes/tree/master/
> > > examples/spark
> > > > >>
> > > > >>
> > > > > --
> > > > > Jean-Baptiste Onofré
> > > > > jbonofre@apache.org
> > > > > http://blog.nanthrax.net
> > > > > Talend - http://www.talend.com
> > > > >
> > > >
> > >
> >
>
Re: Beam on Kubernetes
Posted by Nitin Lamba <nl...@apache.org>.
@Ekrem
Sure! Would be great to work together on this.
I'm assuming you already have the basic docker setup on your dev machine.
If you're new to kubernetes, the first step is to get a local k8s setup
going - single node (minikube) or multi-node (vagrant). Alternatively, you
can setup a container environment on GCP or AWS. It'll help packaging and
test-driving the images/ pods as they're built.
Let me know if you'd like to chat real time (on Slack) about it.
Thanks,
Nitin
On Mon, Feb 20, 2017 at 9:41 PM Ekrem Aksoy <ek...@gmail.com> wrote:
> I'm also interested in this effort. How can I contribute?
>
>
> Best,
>
> Ekrem
>
> On Tue, Feb 21, 2017 at 4:12 AM, Nitin Lamba <ni...@gmail.com>
> wrote:
>
> > Thanks JB, Davor!
> >
> > Yes, the step-by-step Spark/Mesos guide will certainly help.
> >
> > I'll start with packaging java runner/ examples first and see how far I
> > get. Will create a JIRA and should have an update in the next week or so.
> >
> > Best,
> > Nitin
> >
> > On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org> wrote:
> >
> > > I think these are great ideas for simplifying the getting started
> > > experience across runners -- we'd love a contribution in this space!
> > >
> > > On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofré <
> jb@nanthrax.net>
> > > wrote:
> > >
> > > > Hi Nitin,
> > > >
> > > > It sounds like a good candidate for blog or documentation, or even an
> > > > example.
> > > >
> > > > I have a step-by-step example of a pipeline running on Spark with
> Mesos
> > > > (not yet Kubernetes) if you are interested.
> > > >
> > > > Regards
> > > > JB
> > > >
> > > >
> > > > On 02/20/2017 08:47 AM, Nitin Lamba wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Trying to restart this thread from last November [1]. Packaging an
> > > >> end-2-end Beam example for k8s environment, similar to the one from
> > the
> > > >> TensorFlow team [2], may be interesting to look at. The logical
> > > >> progression
> > > >> is:
> > > >>
> > > >> - Start with an example using the local (java) runner
> > > >> - Build the next one for Spark; k8s repo already has Spark v1.5.x as
> > an
> > > >> example [3] that can be updated/ modified
> > > >> - Other runners to follow using Spark as a template
> > > >>
> > > >> Let me know if there is interest in pursuing/ collaborating on this.
> > > >>
> > > >> Thanks,
> > > >> Nitin
> > > >>
> > > >> [1]
> > > >> https://www.mail-archive.com/user@beam.incubator.apache.
> > > org/msg00881.html
> > > >> [2] https://tensorflow.github.io/serving/serving_inception
> > > >> [3] https://github.com/kubernetes/kubernetes/tree/master/
> > examples/spark
> > > >>
> > > >>
> > > > --
> > > > Jean-Baptiste Onofré
> > > > jbonofre@apache.org
> > > > http://blog.nanthrax.net
> > > > Talend - http://www.talend.com
> > > >
> > >
> >
>
Re: Beam on Kubernetes
Posted by Ekrem Aksoy <ek...@gmail.com>.
I'm also interested in this effort. How can I contribute?
Best,
Ekrem
On Tue, Feb 21, 2017 at 4:12 AM, Nitin Lamba <ni...@gmail.com> wrote:
> Thanks JB, Davor!
>
> Yes, the step-by-step Spark/Mesos guide will certainly help.
>
> I'll start with packaging java runner/ examples first and see how far I
> get. Will create a JIRA and should have an update in the next week or so.
>
> Best,
> Nitin
>
> On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org> wrote:
>
> > I think these are great ideas for simplifying the getting started
> > experience across runners -- we'd love a contribution in this space!
> >
> > On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> >
> > > Hi Nitin,
> > >
> > > It sounds like a good candidate for blog or documentation, or even an
> > > example.
> > >
> > > I have a step-by-step example of a pipeline running on Spark with Mesos
> > > (not yet Kubernetes) if you are interested.
> > >
> > > Regards
> > > JB
> > >
> > >
> > > On 02/20/2017 08:47 AM, Nitin Lamba wrote:
> > >
> > >> Hi,
> > >>
> > >> Trying to restart this thread from last November [1]. Packaging an
> > >> end-2-end Beam example for k8s environment, similar to the one from
> the
> > >> TensorFlow team [2], may be interesting to look at. The logical
> > >> progression
> > >> is:
> > >>
> > >> - Start with an example using the local (java) runner
> > >> - Build the next one for Spark; k8s repo already has Spark v1.5.x as
> an
> > >> example [3] that can be updated/ modified
> > >> - Other runners to follow using Spark as a template
> > >>
> > >> Let me know if there is interest in pursuing/ collaborating on this.
> > >>
> > >> Thanks,
> > >> Nitin
> > >>
> > >> [1]
> > >> https://www.mail-archive.com/user@beam.incubator.apache.
> > org/msg00881.html
> > >> [2] https://tensorflow.github.io/serving/serving_inception
> > >> [3] https://github.com/kubernetes/kubernetes/tree/master/
> examples/spark
> > >>
> > >>
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
> >
>
Re: Beam on Kubernetes
Posted by Nitin Lamba <ni...@gmail.com>.
Thanks JB, Davor!
Yes, the step-by-step Spark/Mesos guide will certainly help.
I'll start with packaging java runner/ examples first and see how far I
get. Will create a JIRA and should have an update in the next week or so.
Best,
Nitin
On Mon, Feb 20, 2017 at 2:05 PM, Davor Bonaci <da...@apache.org> wrote:
> I think these are great ideas for simplifying the getting started
> experience across runners -- we'd love a contribution in this space!
>
> On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
> > Hi Nitin,
> >
> > It sounds like a good candidate for blog or documentation, or even an
> > example.
> >
> > I have a step-by-step example of a pipeline running on Spark with Mesos
> > (not yet Kubernetes) if you are interested.
> >
> > Regards
> > JB
> >
> >
> > On 02/20/2017 08:47 AM, Nitin Lamba wrote:
> >
> >> Hi,
> >>
> >> Trying to restart this thread from last November [1]. Packaging an
> >> end-2-end Beam example for k8s environment, similar to the one from the
> >> TensorFlow team [2], may be interesting to look at. The logical
> >> progression
> >> is:
> >>
> >> - Start with an example using the local (java) runner
> >> - Build the next one for Spark; k8s repo already has Spark v1.5.x as an
> >> example [3] that can be updated/ modified
> >> - Other runners to follow using Spark as a template
> >>
> >> Let me know if there is interest in pursuing/ collaborating on this.
> >>
> >> Thanks,
> >> Nitin
> >>
> >> [1]
> >> https://www.mail-archive.com/user@beam.incubator.apache.
> org/msg00881.html
> >> [2] https://tensorflow.github.io/serving/serving_inception
> >> [3] https://github.com/kubernetes/kubernetes/tree/master/examples/spark
> >>
> >>
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
Re: Beam on Kubernetes
Posted by Davor Bonaci <da...@apache.org>.
I think these are great ideas for simplifying the getting started
experience across runners -- we'd love a contribution in this space!
On Mon, Feb 20, 2017 at 12:46 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Nitin,
>
> It sounds like a good candidate for blog or documentation, or even an
> example.
>
> I have a step-by-step example of a pipeline running on Spark with Mesos
> (not yet Kubernetes) if you are interested.
>
> Regards
> JB
>
>
> On 02/20/2017 08:47 AM, Nitin Lamba wrote:
>
>> Hi,
>>
>> Trying to restart this thread from last November [1]. Packaging an
>> end-2-end Beam example for k8s environment, similar to the one from the
>> TensorFlow team [2], may be interesting to look at. The logical
>> progression
>> is:
>>
>> - Start with an example using the local (java) runner
>> - Build the next one for Spark; k8s repo already has Spark v1.5.x as an
>> example [3] that can be updated/ modified
>> - Other runners to follow using Spark as a template
>>
>> Let me know if there is interest in pursuing/ collaborating on this.
>>
>> Thanks,
>> Nitin
>>
>> [1]
>> https://www.mail-archive.com/user@beam.incubator.apache.org/msg00881.html
>> [2] https://tensorflow.github.io/serving/serving_inception
>> [3] https://github.com/kubernetes/kubernetes/tree/master/examples/spark
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Beam on Kubernetes
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Nitin,
It sounds like a good candidate for blog or documentation, or even an
example.
I have a step-by-step example of a pipeline running on Spark with Mesos
(not yet Kubernetes) if you are interested.
Regards
JB
On 02/20/2017 08:47 AM, Nitin Lamba wrote:
> Hi,
>
> Trying to restart this thread from last November [1]. Packaging an
> end-2-end Beam example for k8s environment, similar to the one from the
> TensorFlow team [2], may be interesting to look at. The logical progression
> is:
>
> - Start with an example using the local (java) runner
> - Build the next one for Spark; k8s repo already has Spark v1.5.x as an
> example [3] that can be updated/ modified
> - Other runners to follow using Spark as a template
>
> Let me know if there is interest in pursuing/ collaborating on this.
>
> Thanks,
> Nitin
>
> [1]
> https://www.mail-archive.com/user@beam.incubator.apache.org/msg00881.html
> [2] https://tensorflow.github.io/serving/serving_inception
> [3] https://github.com/kubernetes/kubernetes/tree/master/examples/spark
>
--
Jean-Baptiste Onofr�
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com