You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Timothy Potter <th...@gmail.com> on 2015/05/28 17:23:06 UTC
Confused about exports
This is for the Solr package ... My goal is to expose the ZooKeeper
connection string used by SolrCloud in the registry so client
applications can look it up dynamically. Thus, in my metainfo.xml, I
have the following:
<exportGroups>
<exportGroup>
<name>Servers</name>
<exports>
<export>
<name>host_port</name>
<value>http://${SOLR_HOST}:${site.global.listen_port}/</value>
</export>
<export>
<name>zkhost</name>
<value>${site.global.zk_host}</value>
</export>
</exports>
</exportGroup>
</exportGroups>
<components>
<component>
<name>SOLR</name>
<category>SLAVE</category>
<compExports>Servers-host_port,Servers-zkhost</compExports>
<commandScript>
<script>scripts/solr_node.py</script>
<scriptType>PYTHON</scriptType>
</commandScript>
</component>
</components>
In the slider.log, I see:
2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
agent.AgentProviderService - Attempting to publish zkhost of group
Servers for component type SOLR
2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
agent.AgentProviderService - publishing
PublishedConfiguration{description='Servers' entries = 1}
2015-05-28 09:13:20,371 [1604725103@qtp-819364987-4] INFO
agent.AgentProviderService - Component operation. Status: COMPLETED;
new container state: HEALTHY; new component state: INSTALLED
2015-05-28 09:13:20,371 [AmExecutor-006] INFO
appmaster.SliderAppMaster - Registering component
container_1432823188593_0003_01_000003
When I run the registry command for Solr, I only get the host_port
values, but not the ZooKeeper setting ... the only thing I can think
of is it is the same value for all SOLR components. Here's the output
from the registry command:
[~/dev/lw/projects/incubator-slider/slider-assembly/target/slider-0.81.0-incubating-SNAPSHOT]$
bin/slider registry --name solr --getexp servers
2015-05-28 09:15:49,360 [main] INFO client.RMProxy - Connecting to
ResourceManager at localhost/127.0.0.1:8032
{
"host_port" : [ {
"value" : "http://Lucids-MacBook-Pro.local:49963/",
"containerId" : "container_1432823188593_0003_01_000003",
"tag" : "1",
"level" : "component",
"updatedTime" : "Thu May 28 09:13:20 MDT 2015"
}, {
2015-05-28 09:15:50,966 [main] INFO util.ExitUtil - Exiting with status 0
"value" : "http://Lucids-MacBook-Pro.local:49964/",
"containerId" : "container_1432823188593_0003_01_000002",
"tag" : "2",
"level" : "component",
"updatedTime" : "Thu May 28 09:13:20 MDT 2015"
} ]
}
This is with the latest develop branch.
Re: Confused about exports
Posted by Gour Saha <gs...@hortonworks.com>.
It can be considered as an enhancement, but this is what happens today.
The exports are displayed only if they have one of the 2 random aspects in
them.
They should have, either an allocated port or an assigned host. For ex:
the variables ${SOLR_HOST} and ${site.global.listen_port} are both
dynamically allocated. The zk_host variable has a hard coded value and
hence is not considered to be export worthy.
Hardcoded values can always be extracted by running slider status <app_id>
But I think, if exporting hard coded values helps an application, then we
should consider exporting them as well. Feel free to file a JIRA.
-Gour
On 5/28/15, 9:28 AM, "Timothy Potter" <th...@gmail.com> wrote:
>Yes, for sure (else Solr wouldn't even run):
>
>"site.global.zk_host": "localhost:2181",
>
>
>On Thu, May 28, 2015 at 10:26 AM, Sumit Mohanty
><sm...@hortonworks.com> wrote:
>> Do you have an entry such as
>>
>> "site.global.zk_host":"${ZK_HOST}",
>>
>> in the app config json provided to the app?
>> ________________________________________
>> From: Timothy Potter <th...@gmail.com>
>> Sent: Thursday, May 28, 2015 8:23 AM
>> To: dev@slider.incubator.apache.org
>> Subject: Confused about exports
>>
>> This is for the Solr package ... My goal is to expose the ZooKeeper
>> connection string used by SolrCloud in the registry so client
>> applications can look it up dynamically. Thus, in my metainfo.xml, I
>> have the following:
>>
>> <exportGroups>
>> <exportGroup>
>> <name>Servers</name>
>> <exports>
>> <export>
>> <name>host_port</name>
>> <value>http://${SOLR_HOST}:${site.global.listen_port}/</value>
>> </export>
>> <export>
>> <name>zkhost</name>
>> <value>${site.global.zk_host}</value>
>> </export>
>> </exports>
>> </exportGroup>
>> </exportGroups>
>>
>> <components>
>> <component>
>> <name>SOLR</name>
>> <category>SLAVE</category>
>> <compExports>Servers-host_port,Servers-zkhost</compExports>
>> <commandScript>
>> <script>scripts/solr_node.py</script>
>> <scriptType>PYTHON</scriptType>
>> </commandScript>
>> </component>
>> </components>
>>
>> In the slider.log, I see:
>>
>> 2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
>> agent.AgentProviderService - Attempting to publish zkhost of group
>> Servers for component type SOLR
>> 2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
>> agent.AgentProviderService - publishing
>> PublishedConfiguration{description='Servers' entries = 1}
>> 2015-05-28 09:13:20,371 [1604725103@qtp-819364987-4] INFO
>> agent.AgentProviderService - Component operation. Status: COMPLETED;
>> new container state: HEALTHY; new component state: INSTALLED
>> 2015-05-28 09:13:20,371 [AmExecutor-006] INFO
>> appmaster.SliderAppMaster - Registering component
>> container_1432823188593_0003_01_000003
>>
>> When I run the registry command for Solr, I only get the host_port
>> values, but not the ZooKeeper setting ... the only thing I can think
>> of is it is the same value for all SOLR components. Here's the output
>> from the registry command:
>>
>>
>>[~/dev/lw/projects/incubator-slider/slider-assembly/target/slider-0.81.0-
>>incubating-SNAPSHOT]$
>> bin/slider registry --name solr --getexp servers
>> 2015-05-28 09:15:49,360 [main] INFO client.RMProxy - Connecting to
>> ResourceManager at localhost/127.0.0.1:8032
>> {
>> "host_port" : [ {
>> "value" : "http://Lucids-MacBook-Pro.local:49963/",
>> "containerId" : "container_1432823188593_0003_01_000003",
>> "tag" : "1",
>> "level" : "component",
>> "updatedTime" : "Thu May 28 09:13:20 MDT 2015"
>> }, {
>> 2015-05-28 09:15:50,966 [main] INFO util.ExitUtil - Exiting with
>>status 0
>> "value" : "http://Lucids-MacBook-Pro.local:49964/",
>> "containerId" : "container_1432823188593_0003_01_000002",
>> "tag" : "2",
>> "level" : "component",
>> "updatedTime" : "Thu May 28 09:13:20 MDT 2015"
>> } ]
>> }
>>
>> This is with the latest develop branch.
>
Re: Confused about exports
Posted by Timothy Potter <th...@gmail.com>.
Yes, for sure (else Solr wouldn't even run):
"site.global.zk_host": "localhost:2181",
On Thu, May 28, 2015 at 10:26 AM, Sumit Mohanty
<sm...@hortonworks.com> wrote:
> Do you have an entry such as
>
> "site.global.zk_host":"${ZK_HOST}",
>
> in the app config json provided to the app?
> ________________________________________
> From: Timothy Potter <th...@gmail.com>
> Sent: Thursday, May 28, 2015 8:23 AM
> To: dev@slider.incubator.apache.org
> Subject: Confused about exports
>
> This is for the Solr package ... My goal is to expose the ZooKeeper
> connection string used by SolrCloud in the registry so client
> applications can look it up dynamically. Thus, in my metainfo.xml, I
> have the following:
>
> <exportGroups>
> <exportGroup>
> <name>Servers</name>
> <exports>
> <export>
> <name>host_port</name>
> <value>http://${SOLR_HOST}:${site.global.listen_port}/</value>
> </export>
> <export>
> <name>zkhost</name>
> <value>${site.global.zk_host}</value>
> </export>
> </exports>
> </exportGroup>
> </exportGroups>
>
> <components>
> <component>
> <name>SOLR</name>
> <category>SLAVE</category>
> <compExports>Servers-host_port,Servers-zkhost</compExports>
> <commandScript>
> <script>scripts/solr_node.py</script>
> <scriptType>PYTHON</scriptType>
> </commandScript>
> </component>
> </components>
>
> In the slider.log, I see:
>
> 2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
> agent.AgentProviderService - Attempting to publish zkhost of group
> Servers for component type SOLR
> 2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
> agent.AgentProviderService - publishing
> PublishedConfiguration{description='Servers' entries = 1}
> 2015-05-28 09:13:20,371 [1604725103@qtp-819364987-4] INFO
> agent.AgentProviderService - Component operation. Status: COMPLETED;
> new container state: HEALTHY; new component state: INSTALLED
> 2015-05-28 09:13:20,371 [AmExecutor-006] INFO
> appmaster.SliderAppMaster - Registering component
> container_1432823188593_0003_01_000003
>
> When I run the registry command for Solr, I only get the host_port
> values, but not the ZooKeeper setting ... the only thing I can think
> of is it is the same value for all SOLR components. Here's the output
> from the registry command:
>
> [~/dev/lw/projects/incubator-slider/slider-assembly/target/slider-0.81.0-incubating-SNAPSHOT]$
> bin/slider registry --name solr --getexp servers
> 2015-05-28 09:15:49,360 [main] INFO client.RMProxy - Connecting to
> ResourceManager at localhost/127.0.0.1:8032
> {
> "host_port" : [ {
> "value" : "http://Lucids-MacBook-Pro.local:49963/",
> "containerId" : "container_1432823188593_0003_01_000003",
> "tag" : "1",
> "level" : "component",
> "updatedTime" : "Thu May 28 09:13:20 MDT 2015"
> }, {
> 2015-05-28 09:15:50,966 [main] INFO util.ExitUtil - Exiting with status 0
> "value" : "http://Lucids-MacBook-Pro.local:49964/",
> "containerId" : "container_1432823188593_0003_01_000002",
> "tag" : "2",
> "level" : "component",
> "updatedTime" : "Thu May 28 09:13:20 MDT 2015"
> } ]
> }
>
> This is with the latest develop branch.
Re: Confused about exports
Posted by Sumit Mohanty <sm...@hortonworks.com>.
Do you have an entry such as
"site.global.zk_host":"${ZK_HOST}",
in the app config json provided to the app?
________________________________________
From: Timothy Potter <th...@gmail.com>
Sent: Thursday, May 28, 2015 8:23 AM
To: dev@slider.incubator.apache.org
Subject: Confused about exports
This is for the Solr package ... My goal is to expose the ZooKeeper
connection string used by SolrCloud in the registry so client
applications can look it up dynamically. Thus, in my metainfo.xml, I
have the following:
<exportGroups>
<exportGroup>
<name>Servers</name>
<exports>
<export>
<name>host_port</name>
<value>http://${SOLR_HOST}:${site.global.listen_port}/</value>
</export>
<export>
<name>zkhost</name>
<value>${site.global.zk_host}</value>
</export>
</exports>
</exportGroup>
</exportGroups>
<components>
<component>
<name>SOLR</name>
<category>SLAVE</category>
<compExports>Servers-host_port,Servers-zkhost</compExports>
<commandScript>
<script>scripts/solr_node.py</script>
<scriptType>PYTHON</scriptType>
</commandScript>
</component>
</components>
In the slider.log, I see:
2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
agent.AgentProviderService - Attempting to publish zkhost of group
Servers for component type SOLR
2015-05-28 09:13:20,370 [1604725103@qtp-819364987-4] INFO
agent.AgentProviderService - publishing
PublishedConfiguration{description='Servers' entries = 1}
2015-05-28 09:13:20,371 [1604725103@qtp-819364987-4] INFO
agent.AgentProviderService - Component operation. Status: COMPLETED;
new container state: HEALTHY; new component state: INSTALLED
2015-05-28 09:13:20,371 [AmExecutor-006] INFO
appmaster.SliderAppMaster - Registering component
container_1432823188593_0003_01_000003
When I run the registry command for Solr, I only get the host_port
values, but not the ZooKeeper setting ... the only thing I can think
of is it is the same value for all SOLR components. Here's the output
from the registry command:
[~/dev/lw/projects/incubator-slider/slider-assembly/target/slider-0.81.0-incubating-SNAPSHOT]$
bin/slider registry --name solr --getexp servers
2015-05-28 09:15:49,360 [main] INFO client.RMProxy - Connecting to
ResourceManager at localhost/127.0.0.1:8032
{
"host_port" : [ {
"value" : "http://Lucids-MacBook-Pro.local:49963/",
"containerId" : "container_1432823188593_0003_01_000003",
"tag" : "1",
"level" : "component",
"updatedTime" : "Thu May 28 09:13:20 MDT 2015"
}, {
2015-05-28 09:15:50,966 [main] INFO util.ExitUtil - Exiting with status 0
"value" : "http://Lucids-MacBook-Pro.local:49964/",
"containerId" : "container_1432823188593_0003_01_000002",
"tag" : "2",
"level" : "component",
"updatedTime" : "Thu May 28 09:13:20 MDT 2015"
} ]
}
This is with the latest develop branch.