You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@eagle.apache.org by "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com> on 2015/12/24 06:20:08 UTC
[bug]eagle docker in mac
Hi,
Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run docker well in mac, there are multiple issues:
1) the 2nd docker run hangs forever in starting eagle service.
2) the 1st docker run can start up ambari, and I can access ambari UI, but I can’t access other ports for example 60010 which is for hbase master info port
Thanks
Edward
Re: [bug]eagle docker in mac
Posted by "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com>.
That is perfect!
But as Hortonworks’ HBASE in docker(or sandbox) is not stable, sometimes
HBase master/regionserver automatic shuts down. That is possible some bugs
with Ambari.
Not sure if new Ambari would solve that problem.
Thanks
Edward
On 12/28/15, 22:39, "覃兆坤" <qi...@gmail.com> wrote:
>Hi Edward:
> Docker in Window ox is running well after I increased processor number
>in my virtualbox. Thank you for finding this bug. Storm related port has
>been mapped (8744), the hdfs related ports will be mapped soon.
>
>Thanks
>
>2015-12-25 15:15 GMT+08:00 覃兆坤 <qi...@gmail.com>:
>
>> I am very glad to hear eagle docker can be run in Mac OX. I reset VM 's
>> memory (about 48G), but it still exists problem (Hive metastore stall
>> error), I will continue to follow eagle docker in Windows and map more
>> ports to host. Thanks, Edward.
>>
>> 2015-12-25 13:42 GMT+08:00 Zhang, Edward (GDI Hadoop)
>><yo...@ebay.com>:
>>
>>> I found the reason for hanging is we should assign enough memory/cpu
>>>to vm
>>> in mac OX, otherwise hadoop related processes like hbase is not able
>>>to be
>>> started.
>>>
>>> we should suggest user assign 6G memory and >4 cpus for VM in mac ox.
>>>
>>> Also I would suggest we export hdfs console port, storm related port to
>>> host machine.
>>>
>>> Anything else is very smooth!
>>>
>>> Thanks
>>> Edward
>>>
>>> On 12/23/15, 22:47, "覃兆坤" <qi...@gmail.com> wrote:
>>>
>>> >In your terminal, use command: "docker exec -it sandbox bash" to go
>>>into
>>> >container.
>>> >
>>> >2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop)
>>><yonzhang@ebay.com
>>> >:
>>> >
>>> >> In Ambari, not all components are correctly started up, so I
>>>manually
>>> >> started region server. But hive, kafka, storm may still are not
>>>live.
>>> >>
>>> >> But looks I can’t attach myself into docker container, so I could
>>>not
>>> >>know
>>> >> why it is failing starting eagle service.
>>> >>
>>> >> Are you able to attach to ambari container?
>>> >>
>>> >> On 12/23/15, 22:34, "覃兆坤" <qi...@gmail.com> wrote:
>>> >>
>>> >> >Hi:
>>> >> > I am working to deploy eagle docker in Windows/Mac. I discover
>>>a
>>> >> >problem in Windows. The Hive Metastore component can not start.
>>>Maybe
>>> >>the
>>> >> >same situation happens in Mac. Now you can access ambari UI, so can
>>> you
>>> >> >check whether the ambari UI shows all the components were installed
>>> >> >successfully(hdfs,hbase,hive,kafka,storm....). If the installation
>>> >>failed
>>> >> >in Hive Metastore start, I think the same problem in Windows and
>>>Mac.
>>> I
>>> >> >will try to follow this.
>>> >> >
>>> >> >Thanks
>>> >> >Zhaokun
>>> >> >
>>> >> >2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
>>> >> >
>>> >> >> I think it should because docker runtime under MacOSX/Windows is
>>> >>using
>>> >> >> delegate architecture but not like native LXC container under
>>>Linux
>>> >>OS,
>>> >> >> which may cause some problem like network/ip, etc.
>>> >> >>
>>> >> >> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
>>> >> >> Instances Inside VM]
>>> >> >>
>>> >> >> Zhaokun is helping investigate the problem.
>>> >> >>
>>> >> >> Thanks,
>>> >> >> Hao
>>> >> >>
>>> >> >>
>>> >> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)"
>>> >><yo...@ebay.com>
>>> >> >> wrote:
>>> >> >>
>>> >> >> >Hi,
>>> >> >> >
>>> >> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t
>>>run
>>> >> >> docker well in mac, there are multiple issues:
>>> >> >> >
>>> >> >> >1) the 2nd docker run hangs forever in starting eagle service.
>>> >> >> >
>>> >> >> >2) the 1st docker run can start up ambari, and I can access
>>>ambari
>>> >>UI,
>>> >> >> but I can’t access other ports for example 60010 which is for
>>>hbase
>>> >> >>master
>>> >> >> info port
>>> >> >> >
>>> >> >> >Thanks
>>> >> >> >Edward
>>> >> >>
>>> >>
>>> >>
>>>
>>>
>>
Re: [bug]eagle docker in mac
Posted by 覃兆坤 <qi...@gmail.com>.
Hi Edward:
Docker in Window ox is running well after I increased processor number
in my virtualbox. Thank you for finding this bug. Storm related port has
been mapped (8744), the hdfs related ports will be mapped soon.
Thanks
2015-12-25 15:15 GMT+08:00 覃兆坤 <qi...@gmail.com>:
> I am very glad to hear eagle docker can be run in Mac OX. I reset VM 's
> memory (about 48G), but it still exists problem (Hive metastore stall
> error), I will continue to follow eagle docker in Windows and map more
> ports to host. Thanks, Edward.
>
> 2015-12-25 13:42 GMT+08:00 Zhang, Edward (GDI Hadoop) <yo...@ebay.com>:
>
>> I found the reason for hanging is we should assign enough memory/cpu to vm
>> in mac OX, otherwise hadoop related processes like hbase is not able to be
>> started.
>>
>> we should suggest user assign 6G memory and >4 cpus for VM in mac ox.
>>
>> Also I would suggest we export hdfs console port, storm related port to
>> host machine.
>>
>> Anything else is very smooth!
>>
>> Thanks
>> Edward
>>
>> On 12/23/15, 22:47, "覃兆坤" <qi...@gmail.com> wrote:
>>
>> >In your terminal, use command: "docker exec -it sandbox bash" to go into
>> >container.
>> >
>> >2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop) <yonzhang@ebay.com
>> >:
>> >
>> >> In Ambari, not all components are correctly started up, so I manually
>> >> started region server. But hive, kafka, storm may still are not live.
>> >>
>> >> But looks I can’t attach myself into docker container, so I could not
>> >>know
>> >> why it is failing starting eagle service.
>> >>
>> >> Are you able to attach to ambari container?
>> >>
>> >> On 12/23/15, 22:34, "覃兆坤" <qi...@gmail.com> wrote:
>> >>
>> >> >Hi:
>> >> > I am working to deploy eagle docker in Windows/Mac. I discover a
>> >> >problem in Windows. The Hive Metastore component can not start. Maybe
>> >>the
>> >> >same situation happens in Mac. Now you can access ambari UI, so can
>> you
>> >> >check whether the ambari UI shows all the components were installed
>> >> >successfully(hdfs,hbase,hive,kafka,storm....). If the installation
>> >>failed
>> >> >in Hive Metastore start, I think the same problem in Windows and Mac.
>> I
>> >> >will try to follow this.
>> >> >
>> >> >Thanks
>> >> >Zhaokun
>> >> >
>> >> >2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
>> >> >
>> >> >> I think it should because docker runtime under MacOSX/Windows is
>> >>using
>> >> >> delegate architecture but not like native LXC container under Linux
>> >>OS,
>> >> >> which may cause some problem like network/ip, etc.
>> >> >>
>> >> >> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
>> >> >> Instances Inside VM]
>> >> >>
>> >> >> Zhaokun is helping investigate the problem.
>> >> >>
>> >> >> Thanks,
>> >> >> Hao
>> >> >>
>> >> >>
>> >> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)"
>> >><yo...@ebay.com>
>> >> >> wrote:
>> >> >>
>> >> >> >Hi,
>> >> >> >
>> >> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
>> >> >> docker well in mac, there are multiple issues:
>> >> >> >
>> >> >> >1) the 2nd docker run hangs forever in starting eagle service.
>> >> >> >
>> >> >> >2) the 1st docker run can start up ambari, and I can access ambari
>> >>UI,
>> >> >> but I can’t access other ports for example 60010 which is for hbase
>> >> >>master
>> >> >> info port
>> >> >> >
>> >> >> >Thanks
>> >> >> >Edward
>> >> >>
>> >>
>> >>
>>
>>
>
Re: [bug]eagle docker in mac
Posted by 覃兆坤 <qi...@gmail.com>.
I am very glad to hear eagle docker can be run in Mac OX. I reset VM 's
memory (about 48G), but it still exists problem (Hive metastore stall
error), I will continue to follow eagle docker in Windows and map more
ports to host. Thanks, Edward.
2015-12-25 13:42 GMT+08:00 Zhang, Edward (GDI Hadoop) <yo...@ebay.com>:
> I found the reason for hanging is we should assign enough memory/cpu to vm
> in mac OX, otherwise hadoop related processes like hbase is not able to be
> started.
>
> we should suggest user assign 6G memory and >4 cpus for VM in mac ox.
>
> Also I would suggest we export hdfs console port, storm related port to
> host machine.
>
> Anything else is very smooth!
>
> Thanks
> Edward
>
> On 12/23/15, 22:47, "覃兆坤" <qi...@gmail.com> wrote:
>
> >In your terminal, use command: "docker exec -it sandbox bash" to go into
> >container.
> >
> >2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop) <yonzhang@ebay.com
> >:
> >
> >> In Ambari, not all components are correctly started up, so I manually
> >> started region server. But hive, kafka, storm may still are not live.
> >>
> >> But looks I can’t attach myself into docker container, so I could not
> >>know
> >> why it is failing starting eagle service.
> >>
> >> Are you able to attach to ambari container?
> >>
> >> On 12/23/15, 22:34, "覃兆坤" <qi...@gmail.com> wrote:
> >>
> >> >Hi:
> >> > I am working to deploy eagle docker in Windows/Mac. I discover a
> >> >problem in Windows. The Hive Metastore component can not start. Maybe
> >>the
> >> >same situation happens in Mac. Now you can access ambari UI, so can you
> >> >check whether the ambari UI shows all the components were installed
> >> >successfully(hdfs,hbase,hive,kafka,storm....). If the installation
> >>failed
> >> >in Hive Metastore start, I think the same problem in Windows and Mac. I
> >> >will try to follow this.
> >> >
> >> >Thanks
> >> >Zhaokun
> >> >
> >> >2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
> >> >
> >> >> I think it should because docker runtime under MacOSX/Windows is
> >>using
> >> >> delegate architecture but not like native LXC container under Linux
> >>OS,
> >> >> which may cause some problem like network/ip, etc.
> >> >>
> >> >> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
> >> >> Instances Inside VM]
> >> >>
> >> >> Zhaokun is helping investigate the problem.
> >> >>
> >> >> Thanks,
> >> >> Hao
> >> >>
> >> >>
> >> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)"
> >><yo...@ebay.com>
> >> >> wrote:
> >> >>
> >> >> >Hi,
> >> >> >
> >> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
> >> >> docker well in mac, there are multiple issues:
> >> >> >
> >> >> >1) the 2nd docker run hangs forever in starting eagle service.
> >> >> >
> >> >> >2) the 1st docker run can start up ambari, and I can access ambari
> >>UI,
> >> >> but I can’t access other ports for example 60010 which is for hbase
> >> >>master
> >> >> info port
> >> >> >
> >> >> >Thanks
> >> >> >Edward
> >> >>
> >>
> >>
>
>
Re: [bug]eagle docker in mac
Posted by "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com>.
I found the reason for hanging is we should assign enough memory/cpu to vm
in mac OX, otherwise hadoop related processes like hbase is not able to be
started.
we should suggest user assign 6G memory and >4 cpus for VM in mac ox.
Also I would suggest we export hdfs console port, storm related port to
host machine.
Anything else is very smooth!
Thanks
Edward
On 12/23/15, 22:47, "覃兆坤" <qi...@gmail.com> wrote:
>In your terminal, use command: "docker exec -it sandbox bash" to go into
>container.
>
>2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop) <yo...@ebay.com>:
>
>> In Ambari, not all components are correctly started up, so I manually
>> started region server. But hive, kafka, storm may still are not live.
>>
>> But looks I can’t attach myself into docker container, so I could not
>>know
>> why it is failing starting eagle service.
>>
>> Are you able to attach to ambari container?
>>
>> On 12/23/15, 22:34, "覃兆坤" <qi...@gmail.com> wrote:
>>
>> >Hi:
>> > I am working to deploy eagle docker in Windows/Mac. I discover a
>> >problem in Windows. The Hive Metastore component can not start. Maybe
>>the
>> >same situation happens in Mac. Now you can access ambari UI, so can you
>> >check whether the ambari UI shows all the components were installed
>> >successfully(hdfs,hbase,hive,kafka,storm....). If the installation
>>failed
>> >in Hive Metastore start, I think the same problem in Windows and Mac. I
>> >will try to follow this.
>> >
>> >Thanks
>> >Zhaokun
>> >
>> >2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
>> >
>> >> I think it should because docker runtime under MacOSX/Windows is
>>using
>> >> delegate architecture but not like native LXC container under Linux
>>OS,
>> >> which may cause some problem like network/ip, etc.
>> >>
>> >> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
>> >> Instances Inside VM]
>> >>
>> >> Zhaokun is helping investigate the problem.
>> >>
>> >> Thanks,
>> >> Hao
>> >>
>> >>
>> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)"
>><yo...@ebay.com>
>> >> wrote:
>> >>
>> >> >Hi,
>> >> >
>> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
>> >> docker well in mac, there are multiple issues:
>> >> >
>> >> >1) the 2nd docker run hangs forever in starting eagle service.
>> >> >
>> >> >2) the 1st docker run can start up ambari, and I can access ambari
>>UI,
>> >> but I can’t access other ports for example 60010 which is for hbase
>> >>master
>> >> info port
>> >> >
>> >> >Thanks
>> >> >Edward
>> >>
>>
>>
Re: [bug]eagle docker in mac
Posted by 覃兆坤 <qi...@gmail.com>.
In your terminal, use command: "docker exec -it sandbox bash" to go into
container.
2015-12-24 14:37 GMT+08:00 Zhang, Edward (GDI Hadoop) <yo...@ebay.com>:
> In Ambari, not all components are correctly started up, so I manually
> started region server. But hive, kafka, storm may still are not live.
>
> But looks I can’t attach myself into docker container, so I could not know
> why it is failing starting eagle service.
>
> Are you able to attach to ambari container?
>
> On 12/23/15, 22:34, "覃兆坤" <qi...@gmail.com> wrote:
>
> >Hi:
> > I am working to deploy eagle docker in Windows/Mac. I discover a
> >problem in Windows. The Hive Metastore component can not start. Maybe the
> >same situation happens in Mac. Now you can access ambari UI, so can you
> >check whether the ambari UI shows all the components were installed
> >successfully(hdfs,hbase,hive,kafka,storm....). If the installation failed
> >in Hive Metastore start, I think the same problem in Windows and Mac. I
> >will try to follow this.
> >
> >Thanks
> >Zhaokun
> >
> >2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
> >
> >> I think it should because docker runtime under MacOSX/Windows is using
> >> delegate architecture but not like native LXC container under Linux OS,
> >> which may cause some problem like network/ip, etc.
> >>
> >> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
> >> Instances Inside VM]
> >>
> >> Zhaokun is helping investigate the problem.
> >>
> >> Thanks,
> >> Hao
> >>
> >>
> >> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com>
> >> wrote:
> >>
> >> >Hi,
> >> >
> >> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
> >> docker well in mac, there are multiple issues:
> >> >
> >> >1) the 2nd docker run hangs forever in starting eagle service.
> >> >
> >> >2) the 1st docker run can start up ambari, and I can access ambari UI,
> >> but I can’t access other ports for example 60010 which is for hbase
> >>master
> >> info port
> >> >
> >> >Thanks
> >> >Edward
> >>
>
>
Re: [bug]eagle docker in mac
Posted by "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com>.
In Ambari, not all components are correctly started up, so I manually
started region server. But hive, kafka, storm may still are not live.
But looks I can’t attach myself into docker container, so I could not know
why it is failing starting eagle service.
Are you able to attach to ambari container?
On 12/23/15, 22:34, "覃兆坤" <qi...@gmail.com> wrote:
>Hi:
> I am working to deploy eagle docker in Windows/Mac. I discover a
>problem in Windows. The Hive Metastore component can not start. Maybe the
>same situation happens in Mac. Now you can access ambari UI, so can you
>check whether the ambari UI shows all the components were installed
>successfully(hdfs,hbase,hive,kafka,storm....). If the installation failed
>in Hive Metastore start, I think the same problem in Windows and Mac. I
>will try to follow this.
>
>Thanks
>Zhaokun
>
>2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
>
>> I think it should because docker runtime under MacOSX/Windows is using
>> delegate architecture but not like native LXC container under Linux OS,
>> which may cause some problem like network/ip, etc.
>>
>> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
>> Instances Inside VM]
>>
>> Zhaokun is helping investigate the problem.
>>
>> Thanks,
>> Hao
>>
>>
>> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com>
>> wrote:
>>
>> >Hi,
>> >
>> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
>> docker well in mac, there are multiple issues:
>> >
>> >1) the 2nd docker run hangs forever in starting eagle service.
>> >
>> >2) the 1st docker run can start up ambari, and I can access ambari UI,
>> but I can’t access other ports for example 60010 which is for hbase
>>master
>> info port
>> >
>> >Thanks
>> >Edward
>>
Re: [bug]eagle docker in mac
Posted by 覃兆坤 <qi...@gmail.com>.
Hi:
I am working to deploy eagle docker in Windows/Mac. I discover a
problem in Windows. The Hive Metastore component can not start. Maybe the
same situation happens in Mac. Now you can access ambari UI, so can you
check whether the ambari UI shows all the components were installed
successfully(hdfs,hbase,hive,kafka,storm....). If the installation failed
in Hive Metastore start, I think the same problem in Windows and Mac. I
will try to follow this.
Thanks
Zhaokun
2015-12-24 14:12 GMT+08:00 Chen, Hao <Ha...@ebay.com>:
> I think it should because docker runtime under MacOSX/Windows is using
> delegate architecture but not like native LXC container under Linux OS,
> which may cause some problem like network/ip, etc.
>
> Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker
> Instances Inside VM]
>
> Zhaokun is helping investigate the problem.
>
> Thanks,
> Hao
>
>
> On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com>
> wrote:
>
> >Hi,
> >
> >Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run
> docker well in mac, there are multiple issues:
> >
> >1) the 2nd docker run hangs forever in starting eagle service.
> >
> >2) the 1st docker run can start up ambari, and I can access ambari UI,
> but I can’t access other ports for example 60010 which is for hbase master
> info port
> >
> >Thanks
> >Edward
>
Re: [bug]eagle docker in mac
Posted by "Chen, Hao" <Ha...@ebay.com>.
I think it should because docker runtime under MacOSX/Windows is using delegate architecture but not like native LXC container under Linux OS, which may cause some problem like network/ip, etc.
Docker Proxy Client under Windows/MacOSX -> VirtualBox VM[Docker Instances Inside VM]
Zhaokun is helping investigate the problem.
Thanks,
Hao
On 12/24/15, 1:20 PM, "Zhang, Edward (GDI Hadoop)" <yo...@ebay.com> wrote:
>Hi,
>
>Based on https://hub.docker.com/r/apacheeagle/sandbox/ I can’t run docker well in mac, there are multiple issues:
>
>1) the 2nd docker run hangs forever in starting eagle service.
>
>2) the 1st docker run can start up ambari, and I can access ambari UI, but I can’t access other ports for example 60010 which is for hbase master info port
>
>Thanks
>Edward