You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Webster Homer <we...@sial.com> on 2018/01/19 22:14:40 UTC
Strange Alias behavior
I just discovered some odd behavior with aliases.
We are in the process of converting over to use aliases in solrcloud. We
have a number of collections that applications have referenced the
collections from when we used standalone solr. So we created alias names to
match the name that the java applications already used.
We still have collections that have the name of the alias.
We also decided to create new aliases for use in our ETL process.
I have 3 collections that have the same configset which is named
b2b-catalog-material
collection 1: b2b-catalog-material
collection 2: b2b-catalog-material-180117
collection 3: b2b-catalog-material-180117T
When the alias, b2b-catalog-material-etl is pointed at b2b-catalog-material
and the alias b2b-catalog-material is pointed to b2b-catalog-material-180117
and we do a data load to b2b-catalog-material-etl
We see data being added to both b2b-catalog-material and
b2b-catalog-material-180117
when I delete the alias b2b-catalog-material then the data stopped loading
into the collection b2b-catalog-material-180117
So it seems that alias resolution is somewhat recursive. I'm surprised that
both collections were being updated.
Is this the intended behavior for aliases? I don't remember seeing this
documented.
This was on a solrcloud running solr 7.2
I haven't checked this in Solr 7.2 but when I created a new collection and
then pointed the alias to it and did a search no data was returned because
there was none to return. So this indicates to me that aliases behave
differently if we're writing to them or reading from them.
--
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.
Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by Webster Homer <we...@sial.com>.
Thanks, Erick.
The main reason that we assigned the alias this way was for consistency.
The consistency of communicating with the administrators that actually
maintain the solr instances in our QA and production clouds. They know very
little about solr. It was useful to have all the interfaces in place. They
will be responsible for the processes that set and delete aliases.
Our intent is to migrate away from the situation where the alias and
collection name could ever be the same.
When we only used the alias for reading there was no problem.
The behavior I didn't expect was that when a different alias
b2b-catalog-material-etl was pointing at the collection
b2b-catalog-material we saw data being added to it, but also to another
collection b2b-catalog-material-180117T where the alias
b2b-catalog-material was pointing.
We will be deleting any alias whose name is the same as a collection name
until we have replaced those collections.
On Sat, Jan 20, 2018 at 12:01 PM, Erick Erickson <er...@gmail.com>
wrote:
> SOLR-11488 is there so we formalize what we intend here. For instance,
> at one point we discovered that you could have an alias pointing to
> collection1,collection2 then delete collection2 say. Solr was happy in
> that configuration but it made no sense. See SOLR-11218.
>
> So I don't know what the eventual resolution will be...
>
> In terms of why one would want an alias and collection to have the
> same name, a common recommendation is to completely re-index when
> making schema changes. By being able to create an alias with the same
> name as a collection, you can do that reindexing and atomically switch
> over without affecting the rest of your code or having any service
> interruption. So it looks like this:
>
> create a new_collection and index to it
> create an alias old_collection->new_collection
> delete old_collection
>
> and there's no service interruption. At very best if you couldn't
> create an alias with the same name as your collection, you'd have to
>
> create new_collection and index to it.
> Shut down all your apps
> delete old_collection
> create alias old_collection->new_collection
> bring all your apps back up.
>
> or
> create new_collection and index to it
> create an alias->new_collection
> rewrite all your apps to use the alias
> when they were all re-written, then delete old_collection.
>
> So it is convenient I think. We haven't moved forward on SOLR-11488
> yet. SOLR-11218 beefed up some testing also so we don't inadvertently
> break things.
>
> Best,
> Erick
>
>
>
>
> On Fri, Jan 19, 2018 at 3:06 PM, Webster Homer <we...@sial.com>
> wrote:
> > It seems like a useful feature, especially for migrating from standalone
> to
> > solrcloud, at least if the precedence of alias to collection is defined
> and
> > enforced.
> >
> > On Fri, Jan 19, 2018 at 5:01 PM, Shawn Heisey <ap...@elyograg.org>
> wrote:
> >
> >> On 1/19/2018 3:53 PM, Webster Homer wrote:
> >>
> >>> I created the alias with an existing collection name because our code
> base
> >>> which was created with stand alone solr was a pain to change. I did
> test
> >>> that the alias took precedence over the collection, when I did a
> search.
> >>>
> >>
> >> The ability to create aliases and collections with the same name is
> viewed
> >> as a bug by some, and probably will be removed in a future version.
> >>
> >> https://issues.apache.org/jira/browse/SOLR-11488
> >>
> >> It doesn't really make sense to have an alias with the same name as a
> >> collection, and the behavior is probably undefined.
> >>
> >> Thanks,
> >> Shawn
> >>
> >
> > --
> >
> >
> > This message and any attachment are confidential and may be privileged or
> > otherwise protected from disclosure. If you are not the intended
> recipient,
> > you must not copy this message or attachment or disclose the contents to
> > any other person. If you have received this transmission in error, please
> > notify the sender immediately and delete the message and any attachment
> > from your system. Merck KGaA, Darmstadt, Germany and any of its
> > subsidiaries do not accept liability for any omissions or errors in this
> > message which may arise as a result of E-Mail-transmission or for damages
> > resulting from any unauthorized changes of the content of this message
> and
> > any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> > subsidiaries do not guarantee that this message is free of viruses and
> does
> > not accept liability for any damages caused by any virus transmitted
> > therewith.
> >
> > Click http://www.emdgroup.com/disclaimer to access the German, French,
> > Spanish and Portuguese versions of this disclaimer.
>
--
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.
Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by Erick Erickson <er...@gmail.com>.
SOLR-11488 is there so we formalize what we intend here. For instance,
at one point we discovered that you could have an alias pointing to
collection1,collection2 then delete collection2 say. Solr was happy in
that configuration but it made no sense. See SOLR-11218.
So I don't know what the eventual resolution will be...
In terms of why one would want an alias and collection to have the
same name, a common recommendation is to completely re-index when
making schema changes. By being able to create an alias with the same
name as a collection, you can do that reindexing and atomically switch
over without affecting the rest of your code or having any service
interruption. So it looks like this:
create a new_collection and index to it
create an alias old_collection->new_collection
delete old_collection
and there's no service interruption. At very best if you couldn't
create an alias with the same name as your collection, you'd have to
create new_collection and index to it.
Shut down all your apps
delete old_collection
create alias old_collection->new_collection
bring all your apps back up.
or
create new_collection and index to it
create an alias->new_collection
rewrite all your apps to use the alias
when they were all re-written, then delete old_collection.
So it is convenient I think. We haven't moved forward on SOLR-11488
yet. SOLR-11218 beefed up some testing also so we don't inadvertently
break things.
Best,
Erick
On Fri, Jan 19, 2018 at 3:06 PM, Webster Homer <we...@sial.com> wrote:
> It seems like a useful feature, especially for migrating from standalone to
> solrcloud, at least if the precedence of alias to collection is defined and
> enforced.
>
> On Fri, Jan 19, 2018 at 5:01 PM, Shawn Heisey <ap...@elyograg.org> wrote:
>
>> On 1/19/2018 3:53 PM, Webster Homer wrote:
>>
>>> I created the alias with an existing collection name because our code base
>>> which was created with stand alone solr was a pain to change. I did test
>>> that the alias took precedence over the collection, when I did a search.
>>>
>>
>> The ability to create aliases and collections with the same name is viewed
>> as a bug by some, and probably will be removed in a future version.
>>
>> https://issues.apache.org/jira/browse/SOLR-11488
>>
>> It doesn't really make sense to have an alias with the same name as a
>> collection, and the behavior is probably undefined.
>>
>> Thanks,
>> Shawn
>>
>
> --
>
>
> This message and any attachment are confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> you must not copy this message or attachment or disclose the contents to
> any other person. If you have received this transmission in error, please
> notify the sender immediately and delete the message and any attachment
> from your system. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not accept liability for any omissions or errors in this
> message which may arise as a result of E-Mail-transmission or for damages
> resulting from any unauthorized changes of the content of this message and
> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not guarantee that this message is free of viruses and does
> not accept liability for any damages caused by any virus transmitted
> therewith.
>
> Click http://www.emdgroup.com/disclaimer to access the German, French,
> Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by Webster Homer <we...@sial.com>.
It seems like a useful feature, especially for migrating from standalone to
solrcloud, at least if the precedence of alias to collection is defined and
enforced.
On Fri, Jan 19, 2018 at 5:01 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 1/19/2018 3:53 PM, Webster Homer wrote:
>
>> I created the alias with an existing collection name because our code base
>> which was created with stand alone solr was a pain to change. I did test
>> that the alias took precedence over the collection, when I did a search.
>>
>
> The ability to create aliases and collections with the same name is viewed
> as a bug by some, and probably will be removed in a future version.
>
> https://issues.apache.org/jira/browse/SOLR-11488
>
> It doesn't really make sense to have an alias with the same name as a
> collection, and the behavior is probably undefined.
>
> Thanks,
> Shawn
>
--
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.
Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/19/2018 3:53 PM, Webster Homer wrote:
> I created the alias with an existing collection name because our code base
> which was created with stand alone solr was a pain to change. I did test
> that the alias took precedence over the collection, when I did a search.
The ability to create aliases and collections with the same name is
viewed as a bug by some, and probably will be removed in a future version.
https://issues.apache.org/jira/browse/SOLR-11488
It doesn't really make sense to have an alias with the same name as a
collection, and the behavior is probably undefined.
Thanks,
Shawn
Re: Strange Alias behavior
Posted by Webster Homer <we...@sial.com>.
I created the alias with an existing collection name because our code base
which was created with stand alone solr was a pain to change. I did test
that the alias took precedence over the collection, when I did a search.
On Fri, Jan 19, 2018 at 4:22 PM, Wenjie Zhang (Jack) <
wenjiezhang2013@gmail.com> wrote:
> Why would you create an alias with an existing collection name?
>
> Sent from my iPhone
>
> > On Jan 19, 2018, at 14:14, Webster Homer <we...@sial.com> wrote:
> >
> > I just discovered some odd behavior with aliases.
> >
> > We are in the process of converting over to use aliases in solrcloud. We
> > have a number of collections that applications have referenced the
> > collections from when we used standalone solr. So we created alias names
> to
> > match the name that the java applications already used.
> >
> > We still have collections that have the name of the alias.
> >
> > We also decided to create new aliases for use in our ETL process.
> > I have 3 collections that have the same configset which is named
> > b2b-catalog-material
> > collection 1: b2b-catalog-material
> > collection 2: b2b-catalog-material-180117
> > collection 3: b2b-catalog-material-180117T
> >
> > When the alias, b2b-catalog-material-etl is pointed at
> b2b-catalog-material
> > and the alias b2b-catalog-material is pointed to
> b2b-catalog-material-180117
> >
> > and we do a data load to b2b-catalog-material-etl
> >
> > We see data being added to both b2b-catalog-material and
> > b2b-catalog-material-180117
> >
> > when I delete the alias b2b-catalog-material then the data stopped
> loading
> > into the collection b2b-catalog-material-180117
> >
> >
> > So it seems that alias resolution is somewhat recursive. I'm surprised
> that
> > both collections were being updated.
> >
> > Is this the intended behavior for aliases? I don't remember seeing this
> > documented.
> > This was on a solrcloud running solr 7.2
> >
> > I haven't checked this in Solr 7.2 but when I created a new collection
> and
> > then pointed the alias to it and did a search no data was returned
> because
> > there was none to return. So this indicates to me that aliases behave
> > differently if we're writing to them or reading from them.
> >
> > --
> >
> >
> > This message and any attachment are confidential and may be privileged or
> > otherwise protected from disclosure. If you are not the intended
> recipient,
> > you must not copy this message or attachment or disclose the contents to
> > any other person. If you have received this transmission in error, please
> > notify the sender immediately and delete the message and any attachment
> > from your system. Merck KGaA, Darmstadt, Germany and any of its
> > subsidiaries do not accept liability for any omissions or errors in this
> > message which may arise as a result of E-Mail-transmission or for damages
> > resulting from any unauthorized changes of the content of this message
> and
> > any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> > subsidiaries do not guarantee that this message is free of viruses and
> does
> > not accept liability for any damages caused by any virus transmitted
> > therewith.
> >
> > Click http://www.emdgroup.com/disclaimer to access the German, French,
> > Spanish and Portuguese versions of this disclaimer.
>
--
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.
Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by "Wenjie Zhang (Jack)" <we...@gmail.com>.
Why would you create an alias with an existing collection name?
Sent from my iPhone
> On Jan 19, 2018, at 14:14, Webster Homer <we...@sial.com> wrote:
>
> I just discovered some odd behavior with aliases.
>
> We are in the process of converting over to use aliases in solrcloud. We
> have a number of collections that applications have referenced the
> collections from when we used standalone solr. So we created alias names to
> match the name that the java applications already used.
>
> We still have collections that have the name of the alias.
>
> We also decided to create new aliases for use in our ETL process.
> I have 3 collections that have the same configset which is named
> b2b-catalog-material
> collection 1: b2b-catalog-material
> collection 2: b2b-catalog-material-180117
> collection 3: b2b-catalog-material-180117T
>
> When the alias, b2b-catalog-material-etl is pointed at b2b-catalog-material
> and the alias b2b-catalog-material is pointed to b2b-catalog-material-180117
>
> and we do a data load to b2b-catalog-material-etl
>
> We see data being added to both b2b-catalog-material and
> b2b-catalog-material-180117
>
> when I delete the alias b2b-catalog-material then the data stopped loading
> into the collection b2b-catalog-material-180117
>
>
> So it seems that alias resolution is somewhat recursive. I'm surprised that
> both collections were being updated.
>
> Is this the intended behavior for aliases? I don't remember seeing this
> documented.
> This was on a solrcloud running solr 7.2
>
> I haven't checked this in Solr 7.2 but when I created a new collection and
> then pointed the alias to it and did a search no data was returned because
> there was none to return. So this indicates to me that aliases behave
> differently if we're writing to them or reading from them.
>
> --
>
>
> This message and any attachment are confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> you must not copy this message or attachment or disclose the contents to
> any other person. If you have received this transmission in error, please
> notify the sender immediately and delete the message and any attachment
> from your system. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not accept liability for any omissions or errors in this
> message which may arise as a result of E-Mail-transmission or for damages
> resulting from any unauthorized changes of the content of this message and
> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not guarantee that this message is free of viruses and does
> not accept liability for any damages caused by any virus transmitted
> therewith.
>
> Click http://www.emdgroup.com/disclaimer to access the German, French,
> Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by Webster Homer <we...@sial.com>.
I don't like that this behavior is not documented.
It appears from this that aliases are recursive (sort of) and that isn't
documented.
On Wed, Jan 24, 2018 at 6:38 AM, alessandro.benedetti <a....@sease.io>
wrote:
> b2b-catalog-material-etl -> b2b-catalog-material
> b2b-catalog-material -> b2b-catalog-material-180117
>
> and we do a data load to b2b-catalog-material-etl
>
> We see data being added to both b2b-catalog-material and
> b2b-catalog-material-180117 -> *in here you wanted just to index in
> b2b-catalog-material-180117 I assume*
>
> when I delete the alias b2b-catalog-material then the data stopped loading
> into the collection b2b-catalog-material-180117 -> *this makes sense as
> you
> deleted the alias so the data will just go the b2b-catalog-material
> collection.*
> Why haven't you deleted the old collection instead? what was the purpose of
> deleting the alias ?
>
> To wrap it up, what is that you don't like ?
> is this bit "We see data being added to both b2b-catalog-material and
> b2b-catalog-material-180117" ?
>
>
>
>
> -----
> ---------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
--
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.
Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Re: Strange Alias behavior
Posted by "alessandro.benedetti" <a....@sease.io>.
b2b-catalog-material-etl -> b2b-catalog-material
b2b-catalog-material -> b2b-catalog-material-180117
and we do a data load to b2b-catalog-material-etl
We see data being added to both b2b-catalog-material and
b2b-catalog-material-180117 -> *in here you wanted just to index in
b2b-catalog-material-180117 I assume*
when I delete the alias b2b-catalog-material then the data stopped loading
into the collection b2b-catalog-material-180117 -> *this makes sense as you
deleted the alias so the data will just go the b2b-catalog-material
collection.*
Why haven't you deleted the old collection instead? what was the purpose of
deleting the alias ?
To wrap it up, what is that you don't like ?
is this bit "We see data being added to both b2b-catalog-material and
b2b-catalog-material-180117" ?
-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html