You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jeff Zhang <zj...@gmail.com> on 2015/12/21 06:35:12 UTC
What is the suggested os platform for developing ambari
Hi all,
I check the wiki page for how to develop in ambari. And wondering which is
the suggested os platform for developing ambari. Current I am in max os.
And if I use mac os for development, do I must install VM to verify the
ambari ? is there any other lightways to do that ? Thanks
https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
--
Best Regards
Jeff Zhang
Re: What is the suggested os platform for developing ambari
Posted by Jonathan Hurley <jh...@hortonworks.com>.
There really isn't any documentation since you can't develop certain features on OSX. As I said earlier, you can't run any of the python on OSX - that's one reason that agents need to be pre-bootstrapped and running. You'll need to launch Ambari Server from your IDE. When launching via the IDE, you'll need to have an ambari.properties file on the classpath. This will override many of the hard coded directories (like recommendation output) that won't work on OSX. See my sample ambari.properties that I had included below.
> On Dec 23, 2015, at 12:55 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
> Thanks Jonathan, sorry for bothering you again. But how to start ambari
> server on OSX. sbin/ambari-server script doesn't work on OSX, I check the
> script, it has some hard code path
> like /var/lib/ambari-server/ambari-env.sh
>
> BTW is there any wiki or tutorial about how to develop ambari ? So that I
> won' need to bother you :)
>
> On Wed, Dec 23, 2015 at 12:36 AM, Jonathan Hurley <jh...@hortonworks.com>
> wrote:
>
>> No, you don't. You'll still need agents running on Linux VMs, but you can
>> do the rest of it from OSX if you'd like. Remember that when running Ambari
>> Server on OSX directly from your IDE, it will not push stack updates down
>> the to the agents automatically. You'll want a script that copies the stack
>> resources to your agents so that when you update your service's python
>> files, you can properly deploy and test them.
>>
>>> On Dec 22, 2015, at 3:34 AM, Jeff Zhang <zj...@gmail.com> wrote:
>>>
>>> Thanks Joanthan. Actually I want to add service to ambari and verify
>>> whether it can be installed correctly. Do I have to build the rpm and try
>>> the rpm on a VM ? Is this the suggested method ?
>>>
>>> On Tue, Dec 22, 2015 at 6:35 AM, Jonathan Hurley <
>> jhurley@hortonworks.com>
>>> wrote:
>>>
>>>> It kind of depends on what you want to develop in Ambari.
>>>>
>>>>
>>>> * Using OSX and natively running in Eclipse is great. The problem is
>>>> that you can't test certain items
>>>> * Kerberos (difference between MIT and Heimdal)
>>>> * Server-side python files won't work
>>>> * Agents must be pre-installed (there is no bootstrapping)
>>>>
>>>>
>>>> * Using a VM environment (such as CentOS) gives the most authentic
>>>> experience.
>>>> * Debugging is slower and more cumbersome
>>>> * Kerberos, python, etc all work
>>>> * For ease of development, you'd want to install Ambari Server on
>> a
>>>> VM but have it point to OSX for Eclipse-generated class files. You can
>>>> accomplish this by editing the serverClassPath.py file and pre-pending
>> the
>>>> shared folder between OSX and your VM which has your class files. You
>> can
>>>> also edit ambari-env.sh in order to setup remote debugging so your IDE
>> can
>>>> connect to the remove process.
>>>>
>>>> So, if you don't want to worry about Kerberos or the server-side python
>>>> scripts, then you can use OSX. Below is from an email I had sent
>> earlier in
>>>> the year. But if you want a more "authentic" Ambari install, complete
>> with
>>>> Kerberos and Python, then you'll want to use a VM and just have it link
>> to
>>>> your generated class files.
>>>>
>>>> - Install postgres
>>>> - Load the source code in your IDE of choice
>>>> - Have your own ambari.properties and ensure to include it on the
>>>> classpath when launching from the IDE. This is probably the hardest
>> part to
>>>> setup since you’ll need to override a ton of properties to get Ambari to
>>>> start on OSX. Here are some examples:
>>>>
>>>> java.home=/usr/jdk64/jdk1.7.0_45
>>>> jdk.name=jdk-7u45-linux-x64.tar.gz
>>>>
>>>> resources.dir=/foo/dev/ambari/resources
>>>>
>>>> server.persistence.inMemory=false
>>>> server.os_family=redhat6
>>>> server.os_type=centos6
>>>>
>>>> # postgres
>>>> server.jdbc.database_name=ambari
>>>> server.jdbc.user.name=ambari-server
>>>>
>>>> server.version.file=/foo/dev/ambari/conf/version
>>>>
>>>> metadata.path=src/main/resources/stacks
>>>> security.server.keys_dir=/foo/dev/ambari/keystore
>>>> security.server.passphrase=DEV
>>>>
>>>>
>>>>
>> shared.resources.dir=/foo/src/ambari/ambari-common/src/main/python/ambari_commons/resources
>>>>
>>>>
>> custom.action.definitions=/foo/src/ambari/ambari-server/src/main/resources/custom_action_definitions
>>>> recommendations.dir=/foo/dev/ambari/stack-recommendations
>>>>
>>>>
>> stackadvisor.script=/foo/src/ambari/ambari-server/src/main/resources/scripts/stack_advisor.py
>>>> webapp.dir=/foo/src/ambari/ambari-web/public
>>>> views.dir=/foo/dev/ambari/views
>>>>
>>>>
>> common.services.path=/foo/src/ambari/ambari-server/src/main/resources/common-services
>>>>
>>>>
>>>>
>>>> On Dec 21, 2015, at 12:35 AM, Jeff Zhang <zjffdu@gmail.com<mailto:
>>>> zjffdu@gmail.com>> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I check the wiki page for how to develop in ambari. And wondering which
>> is
>>>> the suggested os platform for developing ambari. Current I am in max os.
>>>> And if I use mac os for development, do I must install VM to verify the
>>>> ambari ? is there any other lightways to do that ? Thanks
>>>>
>>>>
>>>> https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: What is the suggested os platform for developing ambari
Posted by Jeff Zhang <zj...@gmail.com>.
Thanks Jonathan, sorry for bothering you again. But how to start ambari
server on OSX. sbin/ambari-server script doesn't work on OSX, I check the
script, it has some hard code path
like /var/lib/ambari-server/ambari-env.sh
BTW is there any wiki or tutorial about how to develop ambari ? So that I
won' need to bother you :)
On Wed, Dec 23, 2015 at 12:36 AM, Jonathan Hurley <jh...@hortonworks.com>
wrote:
> No, you don't. You'll still need agents running on Linux VMs, but you can
> do the rest of it from OSX if you'd like. Remember that when running Ambari
> Server on OSX directly from your IDE, it will not push stack updates down
> the to the agents automatically. You'll want a script that copies the stack
> resources to your agents so that when you update your service's python
> files, you can properly deploy and test them.
>
> > On Dec 22, 2015, at 3:34 AM, Jeff Zhang <zj...@gmail.com> wrote:
> >
> > Thanks Joanthan. Actually I want to add service to ambari and verify
> > whether it can be installed correctly. Do I have to build the rpm and try
> > the rpm on a VM ? Is this the suggested method ?
> >
> > On Tue, Dec 22, 2015 at 6:35 AM, Jonathan Hurley <
> jhurley@hortonworks.com>
> > wrote:
> >
> >> It kind of depends on what you want to develop in Ambari.
> >>
> >>
> >> * Using OSX and natively running in Eclipse is great. The problem is
> >> that you can't test certain items
> >> * Kerberos (difference between MIT and Heimdal)
> >> * Server-side python files won't work
> >> * Agents must be pre-installed (there is no bootstrapping)
> >>
> >>
> >> * Using a VM environment (such as CentOS) gives the most authentic
> >> experience.
> >> * Debugging is slower and more cumbersome
> >> * Kerberos, python, etc all work
> >> * For ease of development, you'd want to install Ambari Server on
> a
> >> VM but have it point to OSX for Eclipse-generated class files. You can
> >> accomplish this by editing the serverClassPath.py file and pre-pending
> the
> >> shared folder between OSX and your VM which has your class files. You
> can
> >> also edit ambari-env.sh in order to setup remote debugging so your IDE
> can
> >> connect to the remove process.
> >>
> >> So, if you don't want to worry about Kerberos or the server-side python
> >> scripts, then you can use OSX. Below is from an email I had sent
> earlier in
> >> the year. But if you want a more "authentic" Ambari install, complete
> with
> >> Kerberos and Python, then you'll want to use a VM and just have it link
> to
> >> your generated class files.
> >>
> >> - Install postgres
> >> - Load the source code in your IDE of choice
> >> - Have your own ambari.properties and ensure to include it on the
> >> classpath when launching from the IDE. This is probably the hardest
> part to
> >> setup since you’ll need to override a ton of properties to get Ambari to
> >> start on OSX. Here are some examples:
> >>
> >> java.home=/usr/jdk64/jdk1.7.0_45
> >> jdk.name=jdk-7u45-linux-x64.tar.gz
> >>
> >> resources.dir=/foo/dev/ambari/resources
> >>
> >> server.persistence.inMemory=false
> >> server.os_family=redhat6
> >> server.os_type=centos6
> >>
> >> # postgres
> >> server.jdbc.database_name=ambari
> >> server.jdbc.user.name=ambari-server
> >>
> >> server.version.file=/foo/dev/ambari/conf/version
> >>
> >> metadata.path=src/main/resources/stacks
> >> security.server.keys_dir=/foo/dev/ambari/keystore
> >> security.server.passphrase=DEV
> >>
> >>
> >>
> shared.resources.dir=/foo/src/ambari/ambari-common/src/main/python/ambari_commons/resources
> >>
> >>
> custom.action.definitions=/foo/src/ambari/ambari-server/src/main/resources/custom_action_definitions
> >> recommendations.dir=/foo/dev/ambari/stack-recommendations
> >>
> >>
> stackadvisor.script=/foo/src/ambari/ambari-server/src/main/resources/scripts/stack_advisor.py
> >> webapp.dir=/foo/src/ambari/ambari-web/public
> >> views.dir=/foo/dev/ambari/views
> >>
> >>
> common.services.path=/foo/src/ambari/ambari-server/src/main/resources/common-services
> >>
> >>
> >>
> >> On Dec 21, 2015, at 12:35 AM, Jeff Zhang <zjffdu@gmail.com<mailto:
> >> zjffdu@gmail.com>> wrote:
> >>
> >> Hi all,
> >>
> >> I check the wiki page for how to develop in ambari. And wondering which
> is
> >> the suggested os platform for developing ambari. Current I am in max os.
> >> And if I use mac os for development, do I must install VM to verify the
> >> ambari ? is there any other lightways to do that ? Thanks
> >>
> >>
> >> https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
> >>
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >>
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: What is the suggested os platform for developing ambari
Posted by Jonathan Hurley <jh...@hortonworks.com>.
No, you don't. You'll still need agents running on Linux VMs, but you can do the rest of it from OSX if you'd like. Remember that when running Ambari Server on OSX directly from your IDE, it will not push stack updates down the to the agents automatically. You'll want a script that copies the stack resources to your agents so that when you update your service's python files, you can properly deploy and test them.
> On Dec 22, 2015, at 3:34 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
> Thanks Joanthan. Actually I want to add service to ambari and verify
> whether it can be installed correctly. Do I have to build the rpm and try
> the rpm on a VM ? Is this the suggested method ?
>
> On Tue, Dec 22, 2015 at 6:35 AM, Jonathan Hurley <jh...@hortonworks.com>
> wrote:
>
>> It kind of depends on what you want to develop in Ambari.
>>
>>
>> * Using OSX and natively running in Eclipse is great. The problem is
>> that you can't test certain items
>> * Kerberos (difference between MIT and Heimdal)
>> * Server-side python files won't work
>> * Agents must be pre-installed (there is no bootstrapping)
>>
>>
>> * Using a VM environment (such as CentOS) gives the most authentic
>> experience.
>> * Debugging is slower and more cumbersome
>> * Kerberos, python, etc all work
>> * For ease of development, you'd want to install Ambari Server on a
>> VM but have it point to OSX for Eclipse-generated class files. You can
>> accomplish this by editing the serverClassPath.py file and pre-pending the
>> shared folder between OSX and your VM which has your class files. You can
>> also edit ambari-env.sh in order to setup remote debugging so your IDE can
>> connect to the remove process.
>>
>> So, if you don't want to worry about Kerberos or the server-side python
>> scripts, then you can use OSX. Below is from an email I had sent earlier in
>> the year. But if you want a more "authentic" Ambari install, complete with
>> Kerberos and Python, then you'll want to use a VM and just have it link to
>> your generated class files.
>>
>> - Install postgres
>> - Load the source code in your IDE of choice
>> - Have your own ambari.properties and ensure to include it on the
>> classpath when launching from the IDE. This is probably the hardest part to
>> setup since you’ll need to override a ton of properties to get Ambari to
>> start on OSX. Here are some examples:
>>
>> java.home=/usr/jdk64/jdk1.7.0_45
>> jdk.name=jdk-7u45-linux-x64.tar.gz
>>
>> resources.dir=/foo/dev/ambari/resources
>>
>> server.persistence.inMemory=false
>> server.os_family=redhat6
>> server.os_type=centos6
>>
>> # postgres
>> server.jdbc.database_name=ambari
>> server.jdbc.user.name=ambari-server
>>
>> server.version.file=/foo/dev/ambari/conf/version
>>
>> metadata.path=src/main/resources/stacks
>> security.server.keys_dir=/foo/dev/ambari/keystore
>> security.server.passphrase=DEV
>>
>>
>> shared.resources.dir=/foo/src/ambari/ambari-common/src/main/python/ambari_commons/resources
>>
>> custom.action.definitions=/foo/src/ambari/ambari-server/src/main/resources/custom_action_definitions
>> recommendations.dir=/foo/dev/ambari/stack-recommendations
>>
>> stackadvisor.script=/foo/src/ambari/ambari-server/src/main/resources/scripts/stack_advisor.py
>> webapp.dir=/foo/src/ambari/ambari-web/public
>> views.dir=/foo/dev/ambari/views
>>
>> common.services.path=/foo/src/ambari/ambari-server/src/main/resources/common-services
>>
>>
>>
>> On Dec 21, 2015, at 12:35 AM, Jeff Zhang <zjffdu@gmail.com<mailto:
>> zjffdu@gmail.com>> wrote:
>>
>> Hi all,
>>
>> I check the wiki page for how to develop in ambari. And wondering which is
>> the suggested os platform for developing ambari. Current I am in max os.
>> And if I use mac os for development, do I must install VM to verify the
>> ambari ? is there any other lightways to do that ? Thanks
>>
>>
>> https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: What is the suggested os platform for developing ambari
Posted by Jeff Zhang <zj...@gmail.com>.
Thanks Joanthan. Actually I want to add service to ambari and verify
whether it can be installed correctly. Do I have to build the rpm and try
the rpm on a VM ? Is this the suggested method ?
On Tue, Dec 22, 2015 at 6:35 AM, Jonathan Hurley <jh...@hortonworks.com>
wrote:
> It kind of depends on what you want to develop in Ambari.
>
>
> * Using OSX and natively running in Eclipse is great. The problem is
> that you can't test certain items
> * Kerberos (difference between MIT and Heimdal)
> * Server-side python files won't work
> * Agents must be pre-installed (there is no bootstrapping)
>
>
> * Using a VM environment (such as CentOS) gives the most authentic
> experience.
> * Debugging is slower and more cumbersome
> * Kerberos, python, etc all work
> * For ease of development, you'd want to install Ambari Server on a
> VM but have it point to OSX for Eclipse-generated class files. You can
> accomplish this by editing the serverClassPath.py file and pre-pending the
> shared folder between OSX and your VM which has your class files. You can
> also edit ambari-env.sh in order to setup remote debugging so your IDE can
> connect to the remove process.
>
> So, if you don't want to worry about Kerberos or the server-side python
> scripts, then you can use OSX. Below is from an email I had sent earlier in
> the year. But if you want a more "authentic" Ambari install, complete with
> Kerberos and Python, then you'll want to use a VM and just have it link to
> your generated class files.
>
> - Install postgres
> - Load the source code in your IDE of choice
> - Have your own ambari.properties and ensure to include it on the
> classpath when launching from the IDE. This is probably the hardest part to
> setup since you’ll need to override a ton of properties to get Ambari to
> start on OSX. Here are some examples:
>
> java.home=/usr/jdk64/jdk1.7.0_45
> jdk.name=jdk-7u45-linux-x64.tar.gz
>
> resources.dir=/foo/dev/ambari/resources
>
> server.persistence.inMemory=false
> server.os_family=redhat6
> server.os_type=centos6
>
> # postgres
> server.jdbc.database_name=ambari
> server.jdbc.user.name=ambari-server
>
> server.version.file=/foo/dev/ambari/conf/version
>
> metadata.path=src/main/resources/stacks
> security.server.keys_dir=/foo/dev/ambari/keystore
> security.server.passphrase=DEV
>
>
> shared.resources.dir=/foo/src/ambari/ambari-common/src/main/python/ambari_commons/resources
>
> custom.action.definitions=/foo/src/ambari/ambari-server/src/main/resources/custom_action_definitions
> recommendations.dir=/foo/dev/ambari/stack-recommendations
>
> stackadvisor.script=/foo/src/ambari/ambari-server/src/main/resources/scripts/stack_advisor.py
> webapp.dir=/foo/src/ambari/ambari-web/public
> views.dir=/foo/dev/ambari/views
>
> common.services.path=/foo/src/ambari/ambari-server/src/main/resources/common-services
>
>
>
> On Dec 21, 2015, at 12:35 AM, Jeff Zhang <zjffdu@gmail.com<mailto:
> zjffdu@gmail.com>> wrote:
>
> Hi all,
>
> I check the wiki page for how to develop in ambari. And wondering which is
> the suggested os platform for developing ambari. Current I am in max os.
> And if I use mac os for development, do I must install VM to verify the
> ambari ? is there any other lightways to do that ? Thanks
>
>
> https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: What is the suggested os platform for developing ambari
Posted by Jonathan Hurley <jh...@hortonworks.com>.
It kind of depends on what you want to develop in Ambari.
* Using OSX and natively running in Eclipse is great. The problem is that you can't test certain items
* Kerberos (difference between MIT and Heimdal)
* Server-side python files won't work
* Agents must be pre-installed (there is no bootstrapping)
* Using a VM environment (such as CentOS) gives the most authentic experience.
* Debugging is slower and more cumbersome
* Kerberos, python, etc all work
* For ease of development, you'd want to install Ambari Server on a VM but have it point to OSX for Eclipse-generated class files. You can accomplish this by editing the serverClassPath.py file and pre-pending the shared folder between OSX and your VM which has your class files. You can also edit ambari-env.sh in order to setup remote debugging so your IDE can connect to the remove process.
So, if you don't want to worry about Kerberos or the server-side python scripts, then you can use OSX. Below is from an email I had sent earlier in the year. But if you want a more "authentic" Ambari install, complete with Kerberos and Python, then you'll want to use a VM and just have it link to your generated class files.
- Install postgres
- Load the source code in your IDE of choice
- Have your own ambari.properties and ensure to include it on the classpath when launching from the IDE. This is probably the hardest part to setup since you’ll need to override a ton of properties to get Ambari to start on OSX. Here are some examples:
java.home=/usr/jdk64/jdk1.7.0_45
jdk.name=jdk-7u45-linux-x64.tar.gz
resources.dir=/foo/dev/ambari/resources
server.persistence.inMemory=false
server.os_family=redhat6
server.os_type=centos6
# postgres
server.jdbc.database_name=ambari
server.jdbc.user.name=ambari-server
server.version.file=/foo/dev/ambari/conf/version
metadata.path=src/main/resources/stacks
security.server.keys_dir=/foo/dev/ambari/keystore
security.server.passphrase=DEV
shared.resources.dir=/foo/src/ambari/ambari-common/src/main/python/ambari_commons/resources
custom.action.definitions=/foo/src/ambari/ambari-server/src/main/resources/custom_action_definitions
recommendations.dir=/foo/dev/ambari/stack-recommendations
stackadvisor.script=/foo/src/ambari/ambari-server/src/main/resources/scripts/stack_advisor.py
webapp.dir=/foo/src/ambari/ambari-web/public
views.dir=/foo/dev/ambari/views
common.services.path=/foo/src/ambari/ambari-server/src/main/resources/common-services
On Dec 21, 2015, at 12:35 AM, Jeff Zhang <zj...@gmail.com>> wrote:
Hi all,
I check the wiki page for how to develop in ambari. And wondering which is
the suggested os platform for developing ambari. Current I am in max os.
And if I use mac os for development, do I must install VM to verify the
ambari ? is there any other lightways to do that ? Thanks
https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
--
Best Regards
Jeff Zhang