You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Li Shengmei <li...@ict.ac.cn> on 2012/08/24 05:43:46 UTC
About many user accounts in hadoop platform
Hi, all
There are many users in hadoop platform. Can they install their own
hadoop version on the same clusters platform?
I tried to do this but failed. There exsited a user account and the user
install his hadoop. I create another account and install his hadoop. The
logs display "ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 : Address
already in use". So I change the port no. to 8000, but still failed.
When I "start-all.sh", the namenode can't start, the logs display "ERROR
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:lismhadoop cause:java.net.BindException: Address already in use"
Can anyone give some suggestions?
Thanks,
May
Re: About many user accounts in hadoop platform
Posted by Bertrand Dechoux <de...@gmail.com>.
You might also want to look at Hadoop On Demand.
http://hadoop.apache.org/common/docs/r0.17.0/hod.html
But I would not recommend to make one cluster per user.
Regards
Bertrand
On Fri, Aug 24, 2012 at 5:50 AM, Sonal Goyal <so...@gmail.com> wrote:
> Hi,
>
> Do your users want different versions of Hadoop? Or can they share the
> same hadoop cluster and schedule their jobs? If the latter, Hadoop can be
> configured to run for multiple users, and each user can submit their data
> and jobs to the same cluster. Hence you can maintain a single cluster and
> utilize your resources more efficiently. You can read more here:
>
>
> http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
>
> http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
>
> On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
>
>> Hi, all****
>>
>> There are many users in hadoop platform. Can they install their
>> own hadoop version on the same clusters platform? ****
>>
>> I tried to do this but failed. There exsited a user account and the user
>> install his hadoop. I create another account and install his hadoop. The
>> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
>> Address already in use”. So I change the port no. to 8000, but still
>> failed. ****
>>
>> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
>> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
>> as:lismhadoop cause:java.net.BindException: Address already in use”****
>>
>> Can anyone give some suggestions? ****
>>
>> Thanks,****
>>
>> May****
>>
>
>
--
Bertrand Dechoux
Re: About many user accounts in hadoop platform
Posted by Bertrand Dechoux <de...@gmail.com>.
You might also want to look at Hadoop On Demand.
http://hadoop.apache.org/common/docs/r0.17.0/hod.html
But I would not recommend to make one cluster per user.
Regards
Bertrand
On Fri, Aug 24, 2012 at 5:50 AM, Sonal Goyal <so...@gmail.com> wrote:
> Hi,
>
> Do your users want different versions of Hadoop? Or can they share the
> same hadoop cluster and schedule their jobs? If the latter, Hadoop can be
> configured to run for multiple users, and each user can submit their data
> and jobs to the same cluster. Hence you can maintain a single cluster and
> utilize your resources more efficiently. You can read more here:
>
>
> http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
>
> http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
>
> On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
>
>> Hi, all****
>>
>> There are many users in hadoop platform. Can they install their
>> own hadoop version on the same clusters platform? ****
>>
>> I tried to do this but failed. There exsited a user account and the user
>> install his hadoop. I create another account and install his hadoop. The
>> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
>> Address already in use”. So I change the port no. to 8000, but still
>> failed. ****
>>
>> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
>> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
>> as:lismhadoop cause:java.net.BindException: Address already in use”****
>>
>> Can anyone give some suggestions? ****
>>
>> Thanks,****
>>
>> May****
>>
>
>
--
Bertrand Dechoux
Re: About many user accounts in hadoop platform
Posted by Bertrand Dechoux <de...@gmail.com>.
You might also want to look at Hadoop On Demand.
http://hadoop.apache.org/common/docs/r0.17.0/hod.html
But I would not recommend to make one cluster per user.
Regards
Bertrand
On Fri, Aug 24, 2012 at 5:50 AM, Sonal Goyal <so...@gmail.com> wrote:
> Hi,
>
> Do your users want different versions of Hadoop? Or can they share the
> same hadoop cluster and schedule their jobs? If the latter, Hadoop can be
> configured to run for multiple users, and each user can submit their data
> and jobs to the same cluster. Hence you can maintain a single cluster and
> utilize your resources more efficiently. You can read more here:
>
>
> http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
>
> http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
>
> On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
>
>> Hi, all****
>>
>> There are many users in hadoop platform. Can they install their
>> own hadoop version on the same clusters platform? ****
>>
>> I tried to do this but failed. There exsited a user account and the user
>> install his hadoop. I create another account and install his hadoop. The
>> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
>> Address already in use”. So I change the port no. to 8000, but still
>> failed. ****
>>
>> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
>> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
>> as:lismhadoop cause:java.net.BindException: Address already in use”****
>>
>> Can anyone give some suggestions? ****
>>
>> Thanks,****
>>
>> May****
>>
>
>
--
Bertrand Dechoux
Re: About many user accounts in hadoop platform
Posted by Bertrand Dechoux <de...@gmail.com>.
You might also want to look at Hadoop On Demand.
http://hadoop.apache.org/common/docs/r0.17.0/hod.html
But I would not recommend to make one cluster per user.
Regards
Bertrand
On Fri, Aug 24, 2012 at 5:50 AM, Sonal Goyal <so...@gmail.com> wrote:
> Hi,
>
> Do your users want different versions of Hadoop? Or can they share the
> same hadoop cluster and schedule their jobs? If the latter, Hadoop can be
> configured to run for multiple users, and each user can submit their data
> and jobs to the same cluster. Hence you can maintain a single cluster and
> utilize your resources more efficiently. You can read more here:
>
>
> http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
>
> http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
>
> On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
>
>> Hi, all****
>>
>> There are many users in hadoop platform. Can they install their
>> own hadoop version on the same clusters platform? ****
>>
>> I tried to do this but failed. There exsited a user account and the user
>> install his hadoop. I create another account and install his hadoop. The
>> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
>> Address already in use”. So I change the port no. to 8000, but still
>> failed. ****
>>
>> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
>> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
>> as:lismhadoop cause:java.net.BindException: Address already in use”****
>>
>> Can anyone give some suggestions? ****
>>
>> Thanks,****
>>
>> May****
>>
>
>
--
Bertrand Dechoux
Re: About many user accounts in hadoop platform
Posted by Sonal Goyal <so...@gmail.com>.
Hi,
Do your users want different versions of Hadoop? Or can they share the same
hadoop cluster and schedule their jobs? If the latter, Hadoop can be
configured to run for multiple users, and each user can submit their data
and jobs to the same cluster. Hence you can maintain a single cluster and
utilize your resources more efficiently. You can read more here:
http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
> Hi, all****
>
> There are many users in hadoop platform. Can they install their
> own hadoop version on the same clusters platform? ****
>
> I tried to do this but failed. There exsited a user account and the user
> install his hadoop. I create another account and install his hadoop. The
> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
> Address already in use”. So I change the port no. to 8000, but still
> failed. ****
>
> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:lismhadoop cause:java.net.BindException: Address already in use”****
>
> Can anyone give some suggestions? ****
>
> Thanks,****
>
> May****
>
Re: About many user accounts in hadoop platform
Posted by Sonal Goyal <so...@gmail.com>.
Hi,
Do your users want different versions of Hadoop? Or can they share the same
hadoop cluster and schedule their jobs? If the latter, Hadoop can be
configured to run for multiple users, and each user can submit their data
and jobs to the same cluster. Hence you can maintain a single cluster and
utilize your resources more efficiently. You can read more here:
http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
> Hi, all****
>
> There are many users in hadoop platform. Can they install their
> own hadoop version on the same clusters platform? ****
>
> I tried to do this but failed. There exsited a user account and the user
> install his hadoop. I create another account and install his hadoop. The
> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
> Address already in use”. So I change the port no. to 8000, but still
> failed. ****
>
> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:lismhadoop cause:java.net.BindException: Address already in use”****
>
> Can anyone give some suggestions? ****
>
> Thanks,****
>
> May****
>
Re: About many user accounts in hadoop platform
Posted by Sonal Goyal <so...@gmail.com>.
Hi,
Do your users want different versions of Hadoop? Or can they share the same
hadoop cluster and schedule their jobs? If the latter, Hadoop can be
configured to run for multiple users, and each user can submit their data
and jobs to the same cluster. Hence you can maintain a single cluster and
utilize your resources more efficiently. You can read more here:
http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
> Hi, all****
>
> There are many users in hadoop platform. Can they install their
> own hadoop version on the same clusters platform? ****
>
> I tried to do this but failed. There exsited a user account and the user
> install his hadoop. I create another account and install his hadoop. The
> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
> Address already in use”. So I change the port no. to 8000, but still
> failed. ****
>
> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:lismhadoop cause:java.net.BindException: Address already in use”****
>
> Can anyone give some suggestions? ****
>
> Thanks,****
>
> May****
>
Re: About many user accounts in hadoop platform
Posted by Sonal Goyal <so...@gmail.com>.
Hi,
Do your users want different versions of Hadoop? Or can they share the same
hadoop cluster and schedule their jobs? If the latter, Hadoop can be
configured to run for multiple users, and each user can submit their data
and jobs to the same cluster. Hence you can maintain a single cluster and
utilize your resources more efficiently. You can read more here:
http://www.ibm.com/developerworks/linux/library/os-hadoop-scheduling/index.html
http://www.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Aug 24, 2012 at 9:13 AM, Li Shengmei <li...@ict.ac.cn> wrote:
> Hi, all****
>
> There are many users in hadoop platform. Can they install their
> own hadoop version on the same clusters platform? ****
>
> I tried to do this but failed. There exsited a user account and the user
> install his hadoop. I create another account and install his hadoop. The
> logs display “ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
> java.net.BindException: Problem binding to hadoop01/10.3.1.91:9000 :
> Address already in use”. So I change the port no. to 8000, but still
> failed. ****
>
> When I “start-all.sh”, the namenode can’t start, the logs display “ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:lismhadoop cause:java.net.BindException: Address already in use”****
>
> Can anyone give some suggestions? ****
>
> Thanks,****
>
> May****
>