You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Manoj Samel <ma...@gmail.com> on 2015/12/01 03:12:47 UTC
Questions regarding metainfo.xml & and adding components dynamically
Hi,
Looking at the slider example app memcached README.txt, it seems that it is
required to package metainfo.xml when creating the jmemcached-1.0.0.zip
file. It is not required to package appConfig.json or resources.json (even
the appConfig-default.json or resourses-default.json), as these are given
later as command line parameters
1: When the cluster is created, I can see files
/xxx/.slider/cluster/<cluster name>/app_config.json and resources.json (as
well as internal.json etc.). I could not locate any HDFS file corresponding
to metainfo.xml packaged in the .zip. Is metainfo used to generate any
slider files in HDFS ?
2: I need to add components dynamically to cluster. The # of components and
their names are not known before hand. One way to do is when a component is
to be added, add the component entry on local copy of metainfo.xml,
regenerate the .zip and re-install the package with --replacepkg and then
do a slider upgrade with appConfig and resources file containing the new
component as well. Is there any other way to add a new component where I
can *** avoid *** rebuilding the .zip file ?
Thanks,
Re: Questions regarding metainfo.xml & and adding components dynamically
Posted by Manoj Samel <ma...@gmail.com>.
My question now boils down to a simple one - How do I run "slider create"
specifying --metainfo parameter ? I have removed "application.def"
parameter from appConfig.json and I think I am giving a valid metainfo.json
to --metainfo parameter. It still gives a null pointer when trying to read
the metainfo.
What are the mandatory entries in metainfo.json when supplied with
--metainfo ?
Any thoughts on this will be appreciated.
On Mon, Nov 30, 2015 at 7:01 PM, Manoj Samel <ma...@gmail.com>
wrote:
> I tried slider create <> --metainfo metainfo.json and other flags. It gave
> following trace
>
>
>
> 2015-12-01 02:53:13,110 [main] INFO client.SliderClient - No credentials
> requested
> 2015-12-01 02:53:13,182 [main] INFO agent.AgentUtils - Reading metainfo
> at
> hdfs://manoj540/user/workdayadmin/.slider/cluster/ms_no_meta/appdef/appPkg.zip
> Exception: java.lang.NullPointerException
> 2015-12-01 02:53:13,190 [main] ERROR main.ServiceLauncher - Exception:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.slider.providers.agent.AgentClientProvider.getApplicationTags(AgentClientProvider.java:297)
> at
> org.apache.slider.client.SliderClient.launchApplication(SliderClient.java:2045)
> at
> org.apache.slider.client.SliderClient.startCluster(SliderClient.java:1916)
> at
> org.apache.slider.client.SliderClient.actionCreate(SliderClient.java:752)
> at org.apache.slider.client.SliderClient.exec(SliderClient.java:463)
> at org.apache.slider.client.SliderClient.runService(SliderClient.java:424)
> at
> org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
> at
> org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
> at
> org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
> at org.apache.slider.Slider.main(Slider.java:49)
>
> I am on version **** 0.80 ****
>
> The metainfo.json contents are
>
> {
> "metainfo": {
> "schemaVersion": "2.0",
> "application": {
> "name": "ms_no_meta",
> "comment": "ms_no_meta Memcached service deployed by Slider",
> "version": "1.0.0",
> "type": "YARN-APP",
> "exportedConfigs": "None",
> "minHadoopVersion": "2.6.0",
> "osSpecifics": {
> "osSpecific": {
> "osType": "any",
> "packages": {
> "package": {
> "type": "tarball",
> "name": "files/jmemcached-1.0.0.tar"
> }
> }
> }
> }
> }
> }
> }
>
> Any help will be appreciated ...
>
> Thanks,
>
> On Mon, Nov 30, 2015 at 6:43 PM, Manoj Samel <ma...@gmail.com>
> wrote:
>
>> As a follow up, I did following
>>
>> 1) Build the zip file without metainfo.xml
>> 2) slider install
>> 3) slider create xxx added option --metainfo metainfo.xml
>>
>> The create command gave a stack trace in json processing after reading
>> HDFS file from /user/xxx/.slider/cluster/ms_no_meta/appdef/appPkg.zip
>>
>> 4) Unzipping the HDFS file appPkg.zip shows metainfo.json (!!!) whereas I
>> had given metainfo.xml on command line
>>
>> Question - when using command line option --metainfo, it seems the
>> metainfo should be JSON, not XML ... Can someone confirm this ???
>>
>> The info about this is very sparse ...
>>
>> Thanks again !!!
>>
>> On Mon, Nov 30, 2015 at 6:12 PM, Manoj Samel <ma...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Looking at the slider example app memcached README.txt, it seems that it
>>> is required to package metainfo.xml when creating the jmemcached-1.0.0.zip
>>> file. It is not required to package appConfig.json or resources.json (even
>>> the appConfig-default.json or resourses-default.json), as these are given
>>> later as command line parameters
>>>
>>> 1: When the cluster is created, I can see files
>>> /xxx/.slider/cluster/<cluster name>/app_config.json and resources.json (as
>>> well as internal.json etc.). I could not locate any HDFS file corresponding
>>> to metainfo.xml packaged in the .zip. Is metainfo used to generate any
>>> slider files in HDFS ?
>>>
>>> 2: I need to add components dynamically to cluster. The # of components
>>> and their names are not known before hand. One way to do is when a
>>> component is to be added, add the component entry on local copy of
>>> metainfo.xml, regenerate the .zip and re-install the package
>>> with --replacepkg and then do a slider upgrade with appConfig and resources
>>> file containing the new component as well. Is there any other way to add a
>>> new component where I can *** avoid *** rebuilding the .zip file ?
>>>
>>> Thanks,
>>>
>>
>>
>
Re: Questions regarding metainfo.xml & and adding components dynamically
Posted by Manoj Samel <ma...@gmail.com>.
I tried slider create <> --metainfo metainfo.json and other flags. It gave
following trace
2015-12-01 02:53:13,110 [main] INFO client.SliderClient - No credentials
requested
2015-12-01 02:53:13,182 [main] INFO agent.AgentUtils - Reading metainfo at
hdfs://manoj540/user/workdayadmin/.slider/cluster/ms_no_meta/appdef/appPkg.zip
Exception: java.lang.NullPointerException
2015-12-01 02:53:13,190 [main] ERROR main.ServiceLauncher - Exception:
java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.slider.providers.agent.AgentClientProvider.getApplicationTags(AgentClientProvider.java:297)
at
org.apache.slider.client.SliderClient.launchApplication(SliderClient.java:2045)
at
org.apache.slider.client.SliderClient.startCluster(SliderClient.java:1916)
at org.apache.slider.client.SliderClient.actionCreate(SliderClient.java:752)
at org.apache.slider.client.SliderClient.exec(SliderClient.java:463)
at org.apache.slider.client.SliderClient.runService(SliderClient.java:424)
at
org.apache.slider.core.main.ServiceLauncher.launchService(ServiceLauncher.java:188)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceRobustly(ServiceLauncher.java:475)
at
org.apache.slider.core.main.ServiceLauncher.launchServiceAndExit(ServiceLauncher.java:403)
at
org.apache.slider.core.main.ServiceLauncher.serviceMain(ServiceLauncher.java:630)
at org.apache.slider.Slider.main(Slider.java:49)
I am on version **** 0.80 ****
The metainfo.json contents are
{
"metainfo": {
"schemaVersion": "2.0",
"application": {
"name": "ms_no_meta",
"comment": "ms_no_meta Memcached service deployed by Slider",
"version": "1.0.0",
"type": "YARN-APP",
"exportedConfigs": "None",
"minHadoopVersion": "2.6.0",
"osSpecifics": {
"osSpecific": {
"osType": "any",
"packages": {
"package": {
"type": "tarball",
"name": "files/jmemcached-1.0.0.tar"
}
}
}
}
}
}
}
Any help will be appreciated ...
Thanks,
On Mon, Nov 30, 2015 at 6:43 PM, Manoj Samel <ma...@gmail.com>
wrote:
> As a follow up, I did following
>
> 1) Build the zip file without metainfo.xml
> 2) slider install
> 3) slider create xxx added option --metainfo metainfo.xml
>
> The create command gave a stack trace in json processing after reading
> HDFS file from /user/xxx/.slider/cluster/ms_no_meta/appdef/appPkg.zip
>
> 4) Unzipping the HDFS file appPkg.zip shows metainfo.json (!!!) whereas I
> had given metainfo.xml on command line
>
> Question - when using command line option --metainfo, it seems the
> metainfo should be JSON, not XML ... Can someone confirm this ???
>
> The info about this is very sparse ...
>
> Thanks again !!!
>
> On Mon, Nov 30, 2015 at 6:12 PM, Manoj Samel <ma...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Looking at the slider example app memcached README.txt, it seems that it
>> is required to package metainfo.xml when creating the jmemcached-1.0.0.zip
>> file. It is not required to package appConfig.json or resources.json (even
>> the appConfig-default.json or resourses-default.json), as these are given
>> later as command line parameters
>>
>> 1: When the cluster is created, I can see files
>> /xxx/.slider/cluster/<cluster name>/app_config.json and resources.json (as
>> well as internal.json etc.). I could not locate any HDFS file corresponding
>> to metainfo.xml packaged in the .zip. Is metainfo used to generate any
>> slider files in HDFS ?
>>
>> 2: I need to add components dynamically to cluster. The # of components
>> and their names are not known before hand. One way to do is when a
>> component is to be added, add the component entry on local copy of
>> metainfo.xml, regenerate the .zip and re-install the package
>> with --replacepkg and then do a slider upgrade with appConfig and resources
>> file containing the new component as well. Is there any other way to add a
>> new component where I can *** avoid *** rebuilding the .zip file ?
>>
>> Thanks,
>>
>
>
Re: Questions regarding metainfo.xml & and adding components dynamically
Posted by Manoj Samel <ma...@gmail.com>.
As a follow up, I did following
1) Build the zip file without metainfo.xml
2) slider install
3) slider create xxx added option --metainfo metainfo.xml
The create command gave a stack trace in json processing after reading HDFS
file from /user/xxx/.slider/cluster/ms_no_meta/appdef/appPkg.zip
4) Unzipping the HDFS file appPkg.zip shows metainfo.json (!!!) whereas I
had given metainfo.xml on command line
Question - when using command line option --metainfo, it seems the
metainfo should be JSON, not XML ... Can someone confirm this ???
The info about this is very sparse ...
Thanks again !!!
On Mon, Nov 30, 2015 at 6:12 PM, Manoj Samel <ma...@gmail.com>
wrote:
> Hi,
>
> Looking at the slider example app memcached README.txt, it seems that it
> is required to package metainfo.xml when creating the jmemcached-1.0.0.zip
> file. It is not required to package appConfig.json or resources.json (even
> the appConfig-default.json or resourses-default.json), as these are given
> later as command line parameters
>
> 1: When the cluster is created, I can see files
> /xxx/.slider/cluster/<cluster name>/app_config.json and resources.json (as
> well as internal.json etc.). I could not locate any HDFS file corresponding
> to metainfo.xml packaged in the .zip. Is metainfo used to generate any
> slider files in HDFS ?
>
> 2: I need to add components dynamically to cluster. The # of components
> and their names are not known before hand. One way to do is when a
> component is to be added, add the component entry on local copy of
> metainfo.xml, regenerate the .zip and re-install the package
> with --replacepkg and then do a slider upgrade with appConfig and resources
> file containing the new component as well. Is there any other way to add a
> new component where I can *** avoid *** rebuilding the .zip file ?
>
> Thanks,
>