You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Edward J. Yoon" <ed...@apache.org> on 2011/05/23 04:03:56 UTC
Add new cloud infrastructure
Hi whirr,
I'm considering to contribute to this project, and wanting to add
apache hama (BSP-based computing framework)[1]. Although it is
unpopular now, I'm sure there's lots of latent demand for BSP-based
computing engine.
If you guys are ok with this then leave a comment an let me know what you think.
1. http://wiki.apache.org/hama/
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hi all,
May I ask the other committers' opinion?
On Mon, May 23, 2011 at 11:03 AM, Edward J. Yoon <ed...@apache.org> wrote:
> Hi whirr,
>
> I'm considering to contribute to this project, and wanting to add
> apache hama (BSP-based computing framework)[1]. Although it is
> unpopular now, I'm sure there's lots of latent demand for BSP-based
> computing engine.
>
> If you guys are ok with this then leave a comment an let me know what you think.
>
> 1. http://wiki.apache.org/hama/
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
Could anyone test the remote job submission?
2011/5/25 Edward J. Yoon <ed...@apache.org>:
> I think, we need to add web monitoring tool asap. Rest are good.
>
> Sent from my iPhone
>
> On 2011. 5. 25., at 오후 4:28, "ChiaHung Lin" <ch...@nuk.edu.tw> wrote:
>
>> Do we need to redefine/ add some interfaces in order to collaborate with whirr? Having look at api provided by whirr, it seems like to start/ stop hama through whirr would require api conforming to whirr's interfaces.
>>
>> -----Original message-----
>> From:Edward J. Yoon <ed...@apache.org>
>> To:hama-dev@incubator.apache.org
>> Date:Tue, 24 May 2011 17:44:31 +0900
>> Subject:Re: Add new cloud infrastructure
>>
>> I opened a ticket and wrote some code today.
>>
>> Please watch this issue, https://issues.apache.org/jira/browse/WHIRR-313
>>
>> On Tue, May 24, 2011 at 5:04 PM, Tommaso Teofili
>> <to...@gmail.com> wrote:
>>> very interesting Edward, let us know if you need a hand :-)
>>>
>>> 2011/5/23 Edward J. Yoon <ed...@apache.org>
>>>
>>>> Just FYI
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Andrei Savu <sa...@gmail.com>
>>>> Date: Mon, May 23, 2011 at 6:19 PM
>>>> Subject: Re: Add new cloud infrastructure
>>>> To: whirr-dev@incubator.apache.org
>>>>
>>>>
>>>> Hi Edward,
>>>>
>>>> That would be great! Feel free to open an issue and create a patch.
>>>> You should probably take a look at existing code (e.g. zookeeper,
>>>> elasticsearch) to see how a new service is defined.
>>>>
>>>> Whirr can already deploy the components Hama requires: Hadoop 0.20.x
>>>> and Zookeeper so it shouldn't be that hard to add it.
>>>>
>>>> We have a short guide that explains how to fetch the code and submit a
>>>> patch:
>>>> https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>>>>
>>>> Let me know if you need help. :)
>>>>
>>>> Cheers,
>>>>
>>>> -- Andrei Savu / andreisavu.ro
>>>>
>>>> On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org>
>>>> wrote:
>>>>> Hi whirr,
>>>>>
>>>>> I'm considering to contribute to this project, and wanting to add
>>>>> apache hama (BSP-based computing framework)[1]. Although it is
>>>>> unpopular now, I'm sure there's lots of latent demand for BSP-based
>>>>> computing engine.
>>>>>
>>>>> If you guys are ok with this then leave a comment an let me know what you
>>>> think.
>>>>>
>>>>> 1. http://wiki.apache.org/hama/
>>>>>
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>>> @eddieyoon
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon
>>>> @eddieyoon
>>>>
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>>
>> --
>> ChiaHung Lin
>> Department of Information Management
>> National University of Kaohsiung
>> Taiwan
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
I think, we need to add web monitoring tool asap. Rest are good.
Sent from my iPhone
On 2011. 5. 25., at 오후 4:28, "ChiaHung Lin" <ch...@nuk.edu.tw> wrote:
> Do we need to redefine/ add some interfaces in order to collaborate with whirr? Having look at api provided by whirr, it seems like to start/ stop hama through whirr would require api conforming to whirr's interfaces.
>
> -----Original message-----
> From:Edward J. Yoon <ed...@apache.org>
> To:hama-dev@incubator.apache.org
> Date:Tue, 24 May 2011 17:44:31 +0900
> Subject:Re: Add new cloud infrastructure
>
> I opened a ticket and wrote some code today.
>
> Please watch this issue, https://issues.apache.org/jira/browse/WHIRR-313
>
> On Tue, May 24, 2011 at 5:04 PM, Tommaso Teofili
> <to...@gmail.com> wrote:
>> very interesting Edward, let us know if you need a hand :-)
>>
>> 2011/5/23 Edward J. Yoon <ed...@apache.org>
>>
>>> Just FYI
>>>
>>> ---------- Forwarded message ----------
>>> From: Andrei Savu <sa...@gmail.com>
>>> Date: Mon, May 23, 2011 at 6:19 PM
>>> Subject: Re: Add new cloud infrastructure
>>> To: whirr-dev@incubator.apache.org
>>>
>>>
>>> Hi Edward,
>>>
>>> That would be great! Feel free to open an issue and create a patch.
>>> You should probably take a look at existing code (e.g. zookeeper,
>>> elasticsearch) to see how a new service is defined.
>>>
>>> Whirr can already deploy the components Hama requires: Hadoop 0.20.x
>>> and Zookeeper so it shouldn't be that hard to add it.
>>>
>>> We have a short guide that explains how to fetch the code and submit a
>>> patch:
>>> https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>>>
>>> Let me know if you need help. :)
>>>
>>> Cheers,
>>>
>>> -- Andrei Savu / andreisavu.ro
>>>
>>> On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org>
>>> wrote:
>>>> Hi whirr,
>>>>
>>>> I'm considering to contribute to this project, and wanting to add
>>>> apache hama (BSP-based computing framework)[1]. Although it is
>>>> unpopular now, I'm sure there's lots of latent demand for BSP-based
>>>> computing engine.
>>>>
>>>> If you guys are ok with this then leave a comment an let me know what you
>>> think.
>>>>
>>>> 1. http://wiki.apache.org/hama/
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon
>>>> @eddieyoon
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
>
> --
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
Re: Add new cloud infrastructure
Posted by ChiaHung Lin <ch...@nuk.edu.tw>.
Do we need to redefine/ add some interfaces in order to collaborate with whirr? Having look at api provided by whirr, it seems like to start/ stop hama through whirr would require api conforming to whirr's interfaces.
-----Original message-----
From:Edward J. Yoon <ed...@apache.org>
To:hama-dev@incubator.apache.org
Date:Tue, 24 May 2011 17:44:31 +0900
Subject:Re: Add new cloud infrastructure
I opened a ticket and wrote some code today.
Please watch this issue, https://issues.apache.org/jira/browse/WHIRR-313
On Tue, May 24, 2011 at 5:04 PM, Tommaso Teofili
<to...@gmail.com> wrote:
> very interesting Edward, let us know if you need a hand :-)
>
> 2011/5/23 Edward J. Yoon <ed...@apache.org>
>
>> Just FYI
>>
>> ---------- Forwarded message ----------
>> From: Andrei Savu <sa...@gmail.com>
>> Date: Mon, May 23, 2011 at 6:19 PM
>> Subject: Re: Add new cloud infrastructure
>> To: whirr-dev@incubator.apache.org
>>
>>
>> Hi Edward,
>>
>> That would be great! Feel free to open an issue and create a patch.
>> You should probably take a look at existing code (e.g. zookeeper,
>> elasticsearch) to see how a new service is defined.
>>
>> Whirr can already deploy the components Hama requires: Hadoop 0.20.x
>> and Zookeeper so it shouldn't be that hard to add it.
>>
>> We have a short guide that explains how to fetch the code and submit a
>> patch:
>> https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>>
>> Let me know if you need help. :)
>>
>> Cheers,
>>
>> -- Andrei Savu / andreisavu.ro
>>
>> On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>> > Hi whirr,
>> >
>> > I'm considering to contribute to this project, and wanting to add
>> > apache hama (BSP-based computing framework)[1]. Although it is
>> > unpopular now, I'm sure there's lots of latent demand for BSP-based
>> > computing engine.
>> >
>> > If you guys are ok with this then leave a comment an let me know what you
>> think.
>> >
>> > 1. http://wiki.apache.org/hama/
>> >
>> > --
>> > Best Regards, Edward J. Yoon
>> > @eddieyoon
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
--
Best Regards, Edward J. Yoon
@eddieyoon
--
ChiaHung Lin
Department of Information Management
National University of Kaohsiung
Taiwan
Re: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
I opened a ticket and wrote some code today.
Please watch this issue, https://issues.apache.org/jira/browse/WHIRR-313
On Tue, May 24, 2011 at 5:04 PM, Tommaso Teofili
<to...@gmail.com> wrote:
> very interesting Edward, let us know if you need a hand :-)
>
> 2011/5/23 Edward J. Yoon <ed...@apache.org>
>
>> Just FYI
>>
>> ---------- Forwarded message ----------
>> From: Andrei Savu <sa...@gmail.com>
>> Date: Mon, May 23, 2011 at 6:19 PM
>> Subject: Re: Add new cloud infrastructure
>> To: whirr-dev@incubator.apache.org
>>
>>
>> Hi Edward,
>>
>> That would be great! Feel free to open an issue and create a patch.
>> You should probably take a look at existing code (e.g. zookeeper,
>> elasticsearch) to see how a new service is defined.
>>
>> Whirr can already deploy the components Hama requires: Hadoop 0.20.x
>> and Zookeeper so it shouldn't be that hard to add it.
>>
>> We have a short guide that explains how to fetch the code and submit a
>> patch:
>> https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>>
>> Let me know if you need help. :)
>>
>> Cheers,
>>
>> -- Andrei Savu / andreisavu.ro
>>
>> On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org>
>> wrote:
>> > Hi whirr,
>> >
>> > I'm considering to contribute to this project, and wanting to add
>> > apache hama (BSP-based computing framework)[1]. Although it is
>> > unpopular now, I'm sure there's lots of latent demand for BSP-based
>> > computing engine.
>> >
>> > If you guys are ok with this then leave a comment an let me know what you
>> think.
>> >
>> > 1. http://wiki.apache.org/hama/
>> >
>> > --
>> > Best Regards, Edward J. Yoon
>> > @eddieyoon
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by Tommaso Teofili <to...@gmail.com>.
very interesting Edward, let us know if you need a hand :-)
2011/5/23 Edward J. Yoon <ed...@apache.org>
> Just FYI
>
> ---------- Forwarded message ----------
> From: Andrei Savu <sa...@gmail.com>
> Date: Mon, May 23, 2011 at 6:19 PM
> Subject: Re: Add new cloud infrastructure
> To: whirr-dev@incubator.apache.org
>
>
> Hi Edward,
>
> That would be great! Feel free to open an issue and create a patch.
> You should probably take a look at existing code (e.g. zookeeper,
> elasticsearch) to see how a new service is defined.
>
> Whirr can already deploy the components Hama requires: Hadoop 0.20.x
> and Zookeeper so it shouldn't be that hard to add it.
>
> We have a short guide that explains how to fetch the code and submit a
> patch:
> https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>
> Let me know if you need help. :)
>
> Cheers,
>
> -- Andrei Savu / andreisavu.ro
>
> On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org>
> wrote:
> > Hi whirr,
> >
> > I'm considering to contribute to this project, and wanting to add
> > apache hama (BSP-based computing framework)[1]. Although it is
> > unpopular now, I'm sure there's lots of latent demand for BSP-based
> > computing engine.
> >
> > If you guys are ok with this then leave a comment an let me know what you
> think.
> >
> > 1. http://wiki.apache.org/hama/
> >
> > --
> > Best Regards, Edward J. Yoon
> > @eddieyoon
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
Fwd: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
Just FYI
---------- Forwarded message ----------
From: Andrei Savu <sa...@gmail.com>
Date: Mon, May 23, 2011 at 6:19 PM
Subject: Re: Add new cloud infrastructure
To: whirr-dev@incubator.apache.org
Hi Edward,
That would be great! Feel free to open an issue and create a patch.
You should probably take a look at existing code (e.g. zookeeper,
elasticsearch) to see how a new service is defined.
Whirr can already deploy the components Hama requires: Hadoop 0.20.x
and Zookeeper so it shouldn't be that hard to add it.
We have a short guide that explains how to fetch the code and submit a patch:
https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
Let me know if you need help. :)
Cheers,
-- Andrei Savu / andreisavu.ro
On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org> wrote:
> Hi whirr,
>
> I'm considering to contribute to this project, and wanting to add
> apache hama (BSP-based computing framework)[1]. Although it is
> unpopular now, I'm sure there's lots of latent demand for BSP-based
> computing engine.
>
> If you guys are ok with this then leave a comment an let me know what you think.
>
> 1. http://wiki.apache.org/hama/
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by Andrei Savu <sa...@gmail.com>.
On Fri, May 27, 2011 at 4:12 AM, Edward J. Yoon <ed...@apache.org> wrote:
> BTW, what is the estimated release date for the whirr 0.6?
So far the release cycle was ~2 months so probably we will release 0.6
in August or September.
Re: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
BTW, what is the estimated release date for the whirr 0.6?
On Tue, May 24, 2011 at 4:21 PM, Andrei Savu <sa...@gmail.com> wrote:
> On Tue, May 24, 2011 at 10:05 AM, Edward J. Yoon <ed...@apache.org> wrote:
>> By the way, is it impossible to test a service locally?
>
> You could use the byon (bring your own nodes) provider and a bunch of
> virtual machines (with snapshotting). You can find a .properties file
> in "recipes" (nodes-byon.yaml ad zookeeper-byon.properties).
>
> That should be good enough for early testing. Unfortunately when the
> patch is almost ready we still to make sure it works on EC2 and
> RackspaceCloud.
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by Andrei Savu <sa...@gmail.com>.
On Tue, May 24, 2011 at 10:05 AM, Edward J. Yoon <ed...@apache.org> wrote:
> By the way, is it impossible to test a service locally?
You could use the byon (bring your own nodes) provider and a bunch of
virtual machines (with snapshotting). You can find a .properties file
in "recipes" (nodes-byon.yaml ad zookeeper-byon.properties).
That should be good enough for early testing. Unfortunately when the
patch is almost ready we still to make sure it works on EC2 and
RackspaceCloud.
Re: Add new cloud infrastructure
Posted by "Edward J. Yoon" <ed...@apache.org>.
Thanks!
By the way, is it impossible to test a service locally?
On Mon, May 23, 2011 at 6:19 PM, Andrei Savu <sa...@gmail.com> wrote:
> Hi Edward,
>
> That would be great! Feel free to open an issue and create a patch.
> You should probably take a look at existing code (e.g. zookeeper,
> elasticsearch) to see how a new service is defined.
>
> Whirr can already deploy the components Hama requires: Hadoop 0.20.x
> and Zookeeper so it shouldn't be that hard to add it.
>
> We have a short guide that explains how to fetch the code and submit a patch:
> https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
>
> Let me know if you need help. :)
>
> Cheers,
>
> -- Andrei Savu / andreisavu.ro
>
> On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org> wrote:
>> Hi whirr,
>>
>> I'm considering to contribute to this project, and wanting to add
>> apache hama (BSP-based computing framework)[1]. Although it is
>> unpopular now, I'm sure there's lots of latent demand for BSP-based
>> computing engine.
>>
>> If you guys are ok with this then leave a comment an let me know what you think.
>>
>> 1. http://wiki.apache.org/hama/
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
--
Best Regards, Edward J. Yoon
@eddieyoon
Re: Add new cloud infrastructure
Posted by Andrei Savu <sa...@gmail.com>.
Hi Edward,
That would be great! Feel free to open an issue and create a patch.
You should probably take a look at existing code (e.g. zookeeper,
elasticsearch) to see how a new service is defined.
Whirr can already deploy the components Hama requires: Hadoop 0.20.x
and Zookeeper so it shouldn't be that hard to add it.
We have a short guide that explains how to fetch the code and submit a patch:
https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute
Let me know if you need help. :)
Cheers,
-- Andrei Savu / andreisavu.ro
On Mon, May 23, 2011 at 5:03 AM, Edward J. Yoon <ed...@apache.org> wrote:
> Hi whirr,
>
> I'm considering to contribute to this project, and wanting to add
> apache hama (BSP-based computing framework)[1]. Although it is
> unpopular now, I'm sure there's lots of latent demand for BSP-based
> computing engine.
>
> If you guys are ok with this then leave a comment an let me know what you think.
>
> 1. http://wiki.apache.org/hama/
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>