You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@myriad.apache.org by Javi Roman <jr...@gmail.com> on 2018/04/09 17:34:48 UTC
DC/OS vagrant-libvirt ready
Hi!
I've just created a PR [1] with the fist version of DC/OS (tested
v1.11.0) for Myriad development/deployment at DC/OS.
The cluster created by default is something like that:
$ vagrant status
Current machine states:
bt running (libvirt)
m1 running (libvirt)
a1 running (libvirt)
a2 running (libvirt)
a3 running (libvirt)
a4 running (libvirt)
p1 running (libvirt)
The bootstrap machine (with DC/OS CLI installed). One master node, 4
private agents, and one public.
I haven't tested Myraid yet, because we have different options for it:
- Deploy HDFS as a system service.
- Deploy HDFS using Mesosphere Universe (we have to tune this one, the
requirements don't fit in this small Vagrant cluster).
- Deploy Myriad by means of Marathon and Docker.
We have to test different options for this deployment. One interesting
idea would be create a Myriad Universe with packages for HDFS tuned
for this Vagrant, with upstream Apache Hadoop, and packages for
Myriad. This could be a good point for demos.
[1] https://github.com/apache/incubator-myriad/pull/108
--
Javi Roman
Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog: dataintensive.info
Re: DC/OS vagrant-libvirt ready
Posted by Javi Roman <jr...@gmail.com>.
Hi Juan!
I've just created a JIRA with support for HDFS. Great idea some kind
of examples with spark within vagrant, for testing Myriad.
https://issues.apache.org/jira/browse/MYRIAD-263
--
Javi Roman
Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog: dataintensive.info
On Thu, Apr 12, 2018 at 10:30 AM, Juan P <jp...@gmail.com> wrote:
> Congratulations for the work Javi Roman !!
> It seems to me a very interesting tool to accelerate and test the work of
> the contributors. Personally, I would leave HDFS installed as a service to
> make management and maintenance easier. For demonstrations and tests, I
> think it would be interesting to put a spark-shell and update the README
> with some map-reduce example. What do you think?
>
> On 9 April 2018 at 19:34, Javi Roman <jr...@gmail.com> wrote:
>
>> Hi!
>>
>> I've just created a PR [1] with the fist version of DC/OS (tested
>> v1.11.0) for Myriad development/deployment at DC/OS.
>>
>> The cluster created by default is something like that:
>>
>> $ vagrant status
>> Current machine states:
>>
>> bt running (libvirt)
>> m1 running (libvirt)
>> a1 running (libvirt)
>> a2 running (libvirt)
>> a3 running (libvirt)
>> a4 running (libvirt)
>> p1 running (libvirt)
>>
>> The bootstrap machine (with DC/OS CLI installed). One master node, 4
>> private agents, and one public.
>>
>> I haven't tested Myraid yet, because we have different options for it:
>>
>> - Deploy HDFS as a system service.
>> - Deploy HDFS using Mesosphere Universe (we have to tune this one, the
>> requirements don't fit in this small Vagrant cluster).
>> - Deploy Myriad by means of Marathon and Docker.
>>
>> We have to test different options for this deployment. One interesting
>> idea would be create a Myriad Universe with packages for HDFS tuned
>> for this Vagrant, with upstream Apache Hadoop, and packages for
>> Myriad. This could be a good point for demos.
>>
>>
>> [1] https://github.com/apache/incubator-myriad/pull/108
>> --
>> Javi Roman
>>
>> Twitter: @javiromanrh
>> GitHub: github.com/javiroman
>> Linkedin: es.linkedin.com/in/javiroman
>> Big Data Blog: dataintensive.info
>>
Re: DC/OS vagrant-libvirt ready
Posted by Juan P <jp...@gmail.com>.
Congratulations for the work Javi Roman !!
It seems to me a very interesting tool to accelerate and test the work of
the contributors. Personally, I would leave HDFS installed as a service to
make management and maintenance easier. For demonstrations and tests, I
think it would be interesting to put a spark-shell and update the README
with some map-reduce example. What do you think?
On 9 April 2018 at 19:34, Javi Roman <jr...@gmail.com> wrote:
> Hi!
>
> I've just created a PR [1] with the fist version of DC/OS (tested
> v1.11.0) for Myriad development/deployment at DC/OS.
>
> The cluster created by default is something like that:
>
> $ vagrant status
> Current machine states:
>
> bt running (libvirt)
> m1 running (libvirt)
> a1 running (libvirt)
> a2 running (libvirt)
> a3 running (libvirt)
> a4 running (libvirt)
> p1 running (libvirt)
>
> The bootstrap machine (with DC/OS CLI installed). One master node, 4
> private agents, and one public.
>
> I haven't tested Myraid yet, because we have different options for it:
>
> - Deploy HDFS as a system service.
> - Deploy HDFS using Mesosphere Universe (we have to tune this one, the
> requirements don't fit in this small Vagrant cluster).
> - Deploy Myriad by means of Marathon and Docker.
>
> We have to test different options for this deployment. One interesting
> idea would be create a Myriad Universe with packages for HDFS tuned
> for this Vagrant, with upstream Apache Hadoop, and packages for
> Myriad. This could be a good point for demos.
>
>
> [1] https://github.com/apache/incubator-myriad/pull/108
> --
> Javi Roman
>
> Twitter: @javiromanrh
> GitHub: github.com/javiroman
> Linkedin: es.linkedin.com/in/javiroman
> Big Data Blog: dataintensive.info
>