You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Dennis Gearon <ge...@sbcglobal.net> on 2011/01/02 22:58:41 UTC
uuid, COMB uuid, distributed farms
Planning ahead here.
Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
internatiionally distributed Solr/Database project.
Dennis Gearon
Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
Re: uuid, COMB uuid, distributed farms
Posted by Dennis Gearon <ge...@sbcglobal.net>.
Right, Lance, I meant in the field defintion.
I appreciate your help and direction.
Dennis Gearon
Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
----- Original Message ----
From: Lance Norskog <go...@gmail.com>
To: solr-user@lucene.apache.org
Sent: Tue, January 4, 2011 7:15:07 PM
Subject: Re: uuid, COMB uuid, distributed farms
'NOT NULL' in the schema is 'required=true' in a <field> element.
'Search for NOT NULL' is a little odd: you search for a range and then
negate the search, meaning for documents with nothing in that field.
This standard query does it:
-field:[* TO *]
On Tue, Jan 4, 2011 at 2:49 PM, Dennis Gearon <ge...@sbcglobal.net> wrote:
> Thanks Lance.
>
> I will be generating the COMB style of UUID external to Solr.
> Prevents a lot of index paging during INSERTS on DBs, maby eSolr too.
>
> So I would not use 'NEW' in the following, right?
> Just leave default out?
> Some sort of NOT NULL available in a Solr Schema?
>
> <field name="id" type="uuid" indexed="true" stored="true" default="NEW"/>
> PHP code to make the COMB style of UUID,
> easily adapted to other languages, some solutions already exist:
>
> <code>
> //requires php5_uuid module in PHP
> function make_comb_uuid(){
> uuid_create(&$v4);
> uuid_make($v4, UUID_MAKE_V4);
> uuid_export($v4, UUID_FMT_STR, &$v4String);
> $var=gettimeofday();
> return
> substr($v4String,0,24).substr(dechex($var['sec'].$var['usec']),0,12);
>
> }
>
> </code>
>
> Dennis Gearon
>
>
>
>
> ----- Original Message ----
> From: Lance Norskog <go...@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Tue, January 4, 2011 2:15:32 PM
> Subject: Re: uuid, COMB uuid, distributed farms
>
> http://wiki.apache.org/solr/UniqueKey
>
> On Mon, Jan 3, 2011 at 6:55 PM, pankaj bhatt <pa...@gmail.com> wrote:
>> HI Dennis,
>> I have used UUID in context of an application where an installation id
>> (UUID) is generated by the code. It caters to around 10K users.
>> I have not used it in context of SOLR.
>>
>> / Pankaj Bhatt.
>>
>> On Mon, Jan 3, 2011 at 11:05 PM, Dennis Gearon <ge...@sbcglobal.net>wrote:
>>
>>> Thank you Pankaj.
>>>
>>> How large was your installation of Solr? I'm hoping to get mine to be
>>> multinational and making plans for that as I go. So having unique ids,
>>> UUIDs,
>>> that cover a huge addressable space is a requirement.
>>>
>>> If your's was comparable, how were your replication issues, merging issues,
>>> anthing else related to getting large datasets searchable and unique?
>>>
>>> Dennis Gearon
>>>
>>>
>>> Signature Warning
>>> ----------------
>>> It is always a good idea to learn from your own mistakes. It is usually a
>>> better
>>> idea to learn from others’ mistakes, so you do not have to make them
>>> yourself.
>>> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>>>
>>>
>>> EARTH has a Right To Life,
>>> otherwise we all die.
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: pankaj bhatt <pa...@gmail.com>
>>> To: solr-user@lucene.apache.org; gearond@sbcglobal.ne
>>> Sent: Mon, January 3, 2011 8:55:21 AM
>>> Subject: Re: uuid, COMB uuid, distributed farms
>>>
>>> Hi Dennis,
>>>
>>> I have used UUID's in my project to identify a basic installation of
>>> the client.
>>> Can i be of any help.
>>>
>>> / Pankaj Bhatt.
>>>
>>> On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net>
>>> wrote:
>>>
>>> > Planning ahead here.
>>> >
>>> > Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
>>> > internatiionally distributed Solr/Database project.
>>> >
>>> > Dennis Gearon
>>> >
>>> >
>>> > Signature Warning
>>> > ----------------
>>> > It is always a good idea to learn from your own mistakes. It is usually a
>>> > better
>>> > idea to learn from others’ mistakes, so you do not have to make them
>>> > yourself.
>>> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>>> >
>>> >
>>> > EARTH has a Right To Life,
>>> > otherwise we all die.
>>> >
>>> >
>>>
>>>
>>
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
>
--
Lance Norskog
goksron@gmail.com
Re: uuid, COMB uuid, distributed farms
Posted by Lance Norskog <go...@gmail.com>.
'NOT NULL' in the schema is 'required=true' in a <field> element.
'Search for NOT NULL' is a little odd: you search for a range and then
negate the search, meaning for documents with nothing in that field.
This standard query does it:
-field:[* TO *]
On Tue, Jan 4, 2011 at 2:49 PM, Dennis Gearon <ge...@sbcglobal.net> wrote:
> Thanks Lance.
>
> I will be generating the COMB style of UUID external to Solr.
> Prevents a lot of index paging during INSERTS on DBs, maby eSolr too.
>
> So I would not use 'NEW' in the following, right?
> Just leave default out?
> Some sort of NOT NULL available in a Solr Schema?
>
> <field name="id" type="uuid" indexed="true" stored="true" default="NEW"/>
> PHP code to make the COMB style of UUID,
> easily adapted to other languages, some solutions already exist:
>
> <code>
> //requires php5_uuid module in PHP
> function make_comb_uuid(){
> uuid_create(&$v4);
> uuid_make($v4, UUID_MAKE_V4);
> uuid_export($v4, UUID_FMT_STR, &$v4String);
> $var=gettimeofday();
> return
> substr($v4String,0,24).substr(dechex($var['sec'].$var['usec']),0,12);
>
> }
>
> </code>
>
> Dennis Gearon
>
>
>
>
> ----- Original Message ----
> From: Lance Norskog <go...@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Tue, January 4, 2011 2:15:32 PM
> Subject: Re: uuid, COMB uuid, distributed farms
>
> http://wiki.apache.org/solr/UniqueKey
>
> On Mon, Jan 3, 2011 at 6:55 PM, pankaj bhatt <pa...@gmail.com> wrote:
>> HI Dennis,
>> I have used UUID in context of an application where an installation id
>> (UUID) is generated by the code. It caters to around 10K users.
>> I have not used it in context of SOLR.
>>
>> / Pankaj Bhatt.
>>
>> On Mon, Jan 3, 2011 at 11:05 PM, Dennis Gearon <ge...@sbcglobal.net>wrote:
>>
>>> Thank you Pankaj.
>>>
>>> How large was your installation of Solr? I'm hoping to get mine to be
>>> multinational and making plans for that as I go. So having unique ids,
>>> UUIDs,
>>> that cover a huge addressable space is a requirement.
>>>
>>> If your's was comparable, how were your replication issues, merging issues,
>>> anthing else related to getting large datasets searchable and unique?
>>>
>>> Dennis Gearon
>>>
>>>
>>> Signature Warning
>>> ----------------
>>> It is always a good idea to learn from your own mistakes. It is usually a
>>> better
>>> idea to learn from others’ mistakes, so you do not have to make them
>>> yourself.
>>> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>>>
>>>
>>> EARTH has a Right To Life,
>>> otherwise we all die.
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: pankaj bhatt <pa...@gmail.com>
>>> To: solr-user@lucene.apache.org; gearond@sbcglobal.ne
>>> Sent: Mon, January 3, 2011 8:55:21 AM
>>> Subject: Re: uuid, COMB uuid, distributed farms
>>>
>>> Hi Dennis,
>>>
>>> I have used UUID's in my project to identify a basic installation of
>>> the client.
>>> Can i be of any help.
>>>
>>> / Pankaj Bhatt.
>>>
>>> On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net>
>>> wrote:
>>>
>>> > Planning ahead here.
>>> >
>>> > Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
>>> > internatiionally distributed Solr/Database project.
>>> >
>>> > Dennis Gearon
>>> >
>>> >
>>> > Signature Warning
>>> > ----------------
>>> > It is always a good idea to learn from your own mistakes. It is usually a
>>> > better
>>> > idea to learn from others’ mistakes, so you do not have to make them
>>> > yourself.
>>> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>>> >
>>> >
>>> > EARTH has a Right To Life,
>>> > otherwise we all die.
>>> >
>>> >
>>>
>>>
>>
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
>
--
Lance Norskog
goksron@gmail.com
Re: uuid, COMB uuid, distributed farms
Posted by Dennis Gearon <ge...@sbcglobal.net>.
Thanks Lance.
I will be generating the COMB style of UUID external to Solr.
Prevents a lot of index paging during INSERTS on DBs, maby eSolr too.
So I would not use 'NEW' in the following, right?
Just leave default out?
Some sort of NOT NULL available in a Solr Schema?
<field name="id" type="uuid" indexed="true" stored="true" default="NEW"/>
PHP code to make the COMB style of UUID,
easily adapted to other languages, some solutions already exist:
<code>
//requires php5_uuid module in PHP
function make_comb_uuid(){
uuid_create(&$v4);
uuid_make($v4, UUID_MAKE_V4);
uuid_export($v4, UUID_FMT_STR, &$v4String);
$var=gettimeofday();
return
substr($v4String,0,24).substr(dechex($var['sec'].$var['usec']),0,12);
}
</code>
Dennis Gearon
----- Original Message ----
From: Lance Norskog <go...@gmail.com>
To: solr-user@lucene.apache.org
Sent: Tue, January 4, 2011 2:15:32 PM
Subject: Re: uuid, COMB uuid, distributed farms
http://wiki.apache.org/solr/UniqueKey
On Mon, Jan 3, 2011 at 6:55 PM, pankaj bhatt <pa...@gmail.com> wrote:
> HI Dennis,
> I have used UUID in context of an application where an installation id
> (UUID) is generated by the code. It caters to around 10K users.
> I have not used it in context of SOLR.
>
> / Pankaj Bhatt.
>
> On Mon, Jan 3, 2011 at 11:05 PM, Dennis Gearon <ge...@sbcglobal.net>wrote:
>
>> Thank you Pankaj.
>>
>> How large was your installation of Solr? I'm hoping to get mine to be
>> multinational and making plans for that as I go. So having unique ids,
>> UUIDs,
>> that cover a huge addressable space is a requirement.
>>
>> If your's was comparable, how were your replication issues, merging issues,
>> anthing else related to getting large datasets searchable and unique?
>>
>> Dennis Gearon
>>
>>
>> Signature Warning
>> ----------------
>> It is always a good idea to learn from your own mistakes. It is usually a
>> better
>> idea to learn from others’ mistakes, so you do not have to make them
>> yourself.
>> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>>
>>
>> EARTH has a Right To Life,
>> otherwise we all die.
>>
>>
>>
>> ----- Original Message ----
>> From: pankaj bhatt <pa...@gmail.com>
>> To: solr-user@lucene.apache.org; gearond@sbcglobal.ne
>> Sent: Mon, January 3, 2011 8:55:21 AM
>> Subject: Re: uuid, COMB uuid, distributed farms
>>
>> Hi Dennis,
>>
>> I have used UUID's in my project to identify a basic installation of
>> the client.
>> Can i be of any help.
>>
>> / Pankaj Bhatt.
>>
>> On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net>
>> wrote:
>>
>> > Planning ahead here.
>> >
>> > Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
>> > internatiionally distributed Solr/Database project.
>> >
>> > Dennis Gearon
>> >
>> >
>> > Signature Warning
>> > ----------------
>> > It is always a good idea to learn from your own mistakes. It is usually a
>> > better
>> > idea to learn from others’ mistakes, so you do not have to make them
>> > yourself.
>> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>> >
>> >
>> > EARTH has a Right To Life,
>> > otherwise we all die.
>> >
>> >
>>
>>
>
--
Lance Norskog
goksron@gmail.com
Re: uuid, COMB uuid, distributed farms
Posted by Lance Norskog <go...@gmail.com>.
http://wiki.apache.org/solr/UniqueKey
On Mon, Jan 3, 2011 at 6:55 PM, pankaj bhatt <pa...@gmail.com> wrote:
> HI Dennis,
> I have used UUID in context of an application where an installation id
> (UUID) is generated by the code. It caters to around 10K users.
> I have not used it in context of SOLR.
>
> / Pankaj Bhatt.
>
> On Mon, Jan 3, 2011 at 11:05 PM, Dennis Gearon <ge...@sbcglobal.net>wrote:
>
>> Thank you Pankaj.
>>
>> How large was your installation of Solr? I'm hoping to get mine to be
>> multinational and making plans for that as I go. So having unique ids,
>> UUIDs,
>> that cover a huge addressable space is a requirement.
>>
>> If your's was comparable, how were your replication issues, merging issues,
>> anthing else related to getting large datasets searchable and unique?
>>
>> Dennis Gearon
>>
>>
>> Signature Warning
>> ----------------
>> It is always a good idea to learn from your own mistakes. It is usually a
>> better
>> idea to learn from others’ mistakes, so you do not have to make them
>> yourself.
>> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>>
>>
>> EARTH has a Right To Life,
>> otherwise we all die.
>>
>>
>>
>> ----- Original Message ----
>> From: pankaj bhatt <pa...@gmail.com>
>> To: solr-user@lucene.apache.org; gearond@sbcglobal.ne
>> Sent: Mon, January 3, 2011 8:55:21 AM
>> Subject: Re: uuid, COMB uuid, distributed farms
>>
>> Hi Dennis,
>>
>> I have used UUID's in my project to identify a basic installation of
>> the client.
>> Can i be of any help.
>>
>> / Pankaj Bhatt.
>>
>> On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net>
>> wrote:
>>
>> > Planning ahead here.
>> >
>> > Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
>> > internatiionally distributed Solr/Database project.
>> >
>> > Dennis Gearon
>> >
>> >
>> > Signature Warning
>> > ----------------
>> > It is always a good idea to learn from your own mistakes. It is usually a
>> > better
>> > idea to learn from others’ mistakes, so you do not have to make them
>> > yourself.
>> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>> >
>> >
>> > EARTH has a Right To Life,
>> > otherwise we all die.
>> >
>> >
>>
>>
>
--
Lance Norskog
goksron@gmail.com
Re: uuid, COMB uuid, distributed farms
Posted by pankaj bhatt <pa...@gmail.com>.
HI Dennis,
I have used UUID in context of an application where an installation id
(UUID) is generated by the code. It caters to around 10K users.
I have not used it in context of SOLR.
/ Pankaj Bhatt.
On Mon, Jan 3, 2011 at 11:05 PM, Dennis Gearon <ge...@sbcglobal.net>wrote:
> Thank you Pankaj.
>
> How large was your installation of Solr? I'm hoping to get mine to be
> multinational and making plans for that as I go. So having unique ids,
> UUIDs,
> that cover a huge addressable space is a requirement.
>
> If your's was comparable, how were your replication issues, merging issues,
> anthing else related to getting large datasets searchable and unique?
>
> Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
>
> ----- Original Message ----
> From: pankaj bhatt <pa...@gmail.com>
> To: solr-user@lucene.apache.org; gearond@sbcglobal.ne
> Sent: Mon, January 3, 2011 8:55:21 AM
> Subject: Re: uuid, COMB uuid, distributed farms
>
> Hi Dennis,
>
> I have used UUID's in my project to identify a basic installation of
> the client.
> Can i be of any help.
>
> / Pankaj Bhatt.
>
> On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net>
> wrote:
>
> > Planning ahead here.
> >
> > Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
> > internatiionally distributed Solr/Database project.
> >
> > Dennis Gearon
> >
> >
> > Signature Warning
> > ----------------
> > It is always a good idea to learn from your own mistakes. It is usually a
> > better
> > idea to learn from others’ mistakes, so you do not have to make them
> > yourself.
> > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
> >
> >
> > EARTH has a Right To Life,
> > otherwise we all die.
> >
> >
>
>
Re: uuid, COMB uuid, distributed farms
Posted by Dennis Gearon <ge...@sbcglobal.net>.
Thank you Pankaj.
How large was your installation of Solr? I'm hoping to get mine to be
multinational and making plans for that as I go. So having unique ids, UUIDs,
that cover a huge addressable space is a requirement.
If your's was comparable, how were your replication issues, merging issues,
anthing else related to getting large datasets searchable and unique?
Dennis Gearon
Signature Warning
----------------
It is always a good idea to learn from your own mistakes. It is usually a better
idea to learn from others’ mistakes, so you do not have to make them yourself.
from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
----- Original Message ----
From: pankaj bhatt <pa...@gmail.com>
To: solr-user@lucene.apache.org; gearond@sbcglobal.ne
Sent: Mon, January 3, 2011 8:55:21 AM
Subject: Re: uuid, COMB uuid, distributed farms
Hi Dennis,
I have used UUID's in my project to identify a basic installation of
the client.
Can i be of any help.
/ Pankaj Bhatt.
On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net> wrote:
> Planning ahead here.
>
> Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
> internatiionally distributed Solr/Database project.
>
> Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>
Re: uuid, COMB uuid, distributed farms
Posted by pankaj bhatt <pa...@gmail.com>.
Hi Dennis,
I have used UUID's in my project to identify a basic installation of
the client.
Can i be of any help.
/ Pankaj Bhatt.
On Mon, Jan 3, 2011 at 3:28 AM, Dennis Gearon <ge...@sbcglobal.net> wrote:
> Planning ahead here.
>
> Anyone have experience with UUIDs, COMB UUIDs (sequential) in large,
> internatiionally distributed Solr/Database project.
>
> Dennis Gearon
>
>
> Signature Warning
> ----------------
> It is always a good idea to learn from your own mistakes. It is usually a
> better
> idea to learn from others’ mistakes, so you do not have to make them
> yourself.
> from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036'
>
>
> EARTH has a Right To Life,
> otherwise we all die.
>
>