You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Gus Heck (JIRA)" <ji...@apache.org> on 2017/12/31 01:56:00 UTC
[jira] [Updated] (SOLR-11722) API to create a Time Routed Alias and
first collection
[ https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gus Heck updated SOLR-11722:
----------------------------
Attachment: SOLR-11722.patch
A very preliminary patch. Needs tests, needs docs, comments welcome. I had to touch a couple of areas to make code re-usable... specifically the collection creation command code got moved to a method that could also be called for routed alias creation of the initial collection. I also extracted the anonymous inner implementation of SolrParams that was doing the v2 to v1 conversion into an explicit inner class so that I didn't have to duplicate that either.
Having added the TZ parameter as suggested, I suspect there is work to be done in TimeRoutedAliasUpdateProcessor to consume this when parsing collection names, but I I'm not sure if that's within this ticket or should be done elsewhere.
An example that seemed to work just fine for me locally:
{code}
{
"create-routed-alias" : {
"name": "testalias",
"routing-method": "time",
"routed-field": "evt_dt",
"start":"NOW",
"increment":"+2HOUR",
"create-collection" : {
"router": {
"name":"implicit",
"field":"foo_s"
},
"shards":"foo,bar,baz",
"config":"_default",
"numShards": 3,
"tlogReplicas":1,
"pullReplicas":1,
"maxShardsPerNode":2,
"properties" : {
"foobar":"bazbam"
}
}
}
}
{code}
which produced alias metadata of:
{code}
{
"collection":{"testalias":"testalias_2017-12-30_00_32_02"},
"collection_metadata":{"testalias":{
"collection-create.tlogReplicas":"1",
"collection-create.maxShardsPerNode":"2",
"start":"NOW",
"increment":"+2HOUR",
"collection-create.pullReplicas":"1",
"collection-create.collection.configName":"_default",
"collection-create.numShards":"3",
"routing-method":"time",
"routed-field":"evt_dt",
"collection-create.router.name":"implicit",
"collection-create.router.field":"foo_s",
"collection-create.shards":"foo,bar,baz",
"collection-create.property.foobar":"bazbam"}}}
{code}
plus the following collection:
{code}
{
"responseHeader": {
"status": 0,
"QTime": 37
},
"cluster": {
"collections": {
"testalias_2017-12-30_00_32_02": {
"pullReplicas": "1",
"replicationFactor": "1",
"shards": {
"foo": {
"range": null,
"state": "active",
"replicas": {
"core_node3": {
"core": "testalias_2017-12-30_00_32_02_foo_replica_t1",
"base_url": "http://127.0.1.1:8983/solr",
"node_name": "127.0.1.1:8983_solr",
"state": "active",
"type": "TLOG",
"leader": "true"
},
"core_node5": {
"core": "testalias_2017-12-30_00_32_02_foo_replica_p2",
"base_url": "http://127.0.1.1:8984/solr",
"node_name": "127.0.1.1:8984_solr",
"state": "active",
"type": "PULL"
}
}
},
"bar": {
"range": null,
"state": "active",
"replicas": {
"core_node7": {
"core": "testalias_2017-12-30_00_32_02_bar_replica_t4",
"base_url": "http://127.0.1.1:8981/solr",
"node_name": "127.0.1.1:8981_solr",
"state": "active",
"type": "TLOG",
"leader": "true"
},
"core_node9": {
"core": "testalias_2017-12-30_00_32_02_bar_replica_p6",
"base_url": "http://127.0.1.1:8982/solr",
"node_name": "127.0.1.1:8982_solr",
"state": "active",
"type": "PULL"
}
}
},
"baz": {
"range": null,
"state": "active",
"replicas": {
"core_node11": {
"core": "testalias_2017-12-30_00_32_02_baz_replica_t8",
"base_url": "http://127.0.1.1:8983/solr",
"node_name": "127.0.1.1:8983_solr",
"state": "active",
"type": "TLOG",
"leader": "true"
},
"core_node12": {
"core": "testalias_2017-12-30_00_32_02_baz_replica_p10",
"base_url": "http://127.0.1.1:8984/solr",
"node_name": "127.0.1.1:8984_solr",
"state": "active",
"type": "PULL"
}
}
}
},
"router": {
"field": "foo_s",
"name": "implicit"
},
"maxShardsPerNode": "2",
"autoAddReplicas": "false",
"nrtReplicas": "1",
"tlogReplicas": "1",
"znodeVersion": 29,
"aliases": [
"testalias"
],
"configName": "_default"
}
},
"aliases": {
"testalias": "testalias_2017-12-30_00_32_02"
},
"live_nodes": [
"127.0.1.1:8982_solr",
"127.0.1.1:8984_solr",
"127.0.1.1:8981_solr",
"127.0.1.1:8983_solr"
]
}
}
{code}
> API to create a Time Routed Alias and first collection
> ------------------------------------------------------
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: David Smiley
> Attachments: SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed Alias" along with its first collection. Need to decide what endpoint URL it is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or alternatively piggy-back off of command=create-alias but we add more options, perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias name via a core property), and then the alias pointing to it which demands collections exist first. When figuring the collection name, you'll need to reference the format in TimeRoutedAliasUpdateProcessor.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org