You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sachin Aggarwal <di...@gmail.com> on 2012/07/10 12:08:24 UTC
unique key
is it possible not to index and but store the unique key....it will make
index size small. i need the unique key to b stored so that i can read the
returned result from the database.
--
Thanks & Regards
Sachin Aggarwal
7760502772
Re: unsubsribe
Posted by Erick Erickson <er...@gmail.com>.
This isn't a good idea. You _must_ index the unique key in order for
documents having that unique key to be found and deleted when you
add another document with that uniqueKey.
Best
Erick
On Tue, Jul 10, 2012 at 10:32 AM, Noordeen, Roxy
<Ro...@wwecorp.com> wrote:
>
>
> -----Original Message-----
> From: Sachin Aggarwal [mailto:different.sachin@gmail.com]
> Sent: Tuesday, July 10, 2012 6:08 AM
> To: solr-user@lucene.apache.org
> Subject: unique key
>
> is it possible not to index and but store the unique key....it will make
> index size small. i need the unique key to b stored so that i can read the
> returned result from the database.
>
> --
>
> Thanks & Regards
>
> Sachin Aggarwal
> 7760502772
unsubsribe
Posted by "Noordeen, Roxy" <Ro...@wwecorp.com>.
-----Original Message-----
From: Sachin Aggarwal [mailto:different.sachin@gmail.com]
Sent: Tuesday, July 10, 2012 6:08 AM
To: solr-user@lucene.apache.org
Subject: unique key
is it possible not to index and but store the unique key....it will make
index size small. i need the unique key to b stored so that i can read the
returned result from the database.
--
Thanks & Regards
Sachin Aggarwal
7760502772
Re: unique key
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
No, a unique key needs to be indexed. You can delete documents by query (to
avoid duplication), but you can't query on any field that is not indexed,
so I guess you'll need it.
On Tue, Jul 10, 2012 at 12:23 PM, Sachin Aggarwal <
different.sachin@gmail.com> wrote:
> today i experimented some parameters with apache-solr-4.0.0-ALPHA
>
> example record :-
> id,u,u1,u2,u3,u4,u5,u6,u7,u8,u9
> df8caf0b-a0b2-4cc8-8594-e9b17026f126, ARGUE, COUDE, FLONG, JIVER, MALAR,
> PARVO, SARIS, ULANS, VIRLSA, CHATEAUBRIAND
>
>
> for 5000 index dir size is when unique key is not indexed:::::::596.19 KB
> for 5000 index dir size is when unique key is not indexed:::::::399.98 KB
>
> for 5005000 index dir size is when unique key is not indexed:::::333.73 MB
> for 5005000 index dir size is when unique key is indexed:::::::::::520.46
> MB
>
> for 10010000 index dir size is when unique key is not indexed:::::667.42 MB
> for 10010000 index dir size is when unique key is indexed:::::::::::1.02 GB
>
> there is a significant decrease in index size but it costs me duplication
> of record if the job fails.....thats too dangerous is there any way where
> we can make sure that solr maintain unique key but don't index it.
>
Re: unique key
Posted by Sachin Aggarwal <di...@gmail.com>.
today i experimented some parameters with apache-solr-4.0.0-ALPHA
example record :-
id,u,u1,u2,u3,u4,u5,u6,u7,u8,u9
df8caf0b-a0b2-4cc8-8594-e9b17026f126, ARGUE, COUDE, FLONG, JIVER, MALAR,
PARVO, SARIS, ULANS, VIRLSA, CHATEAUBRIAND
for 5000 index dir size is when unique key is not indexed:::::::596.19 KB
for 5000 index dir size is when unique key is not indexed:::::::399.98 KB
for 5005000 index dir size is when unique key is not indexed:::::333.73 MB
for 5005000 index dir size is when unique key is indexed:::::::::::520.46 MB
for 10010000 index dir size is when unique key is not indexed:::::667.42 MB
for 10010000 index dir size is when unique key is indexed:::::::::::1.02 GB
there is a significant decrease in index size but it costs me duplication
of record if the job fails.....thats too dangerous is there any way where
we can make sure that solr maintain unique key but don't index it.
Re: unique key
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
There are some specific use cases where you can skip having a unique key.
See http://wiki.apache.org/solr/UniqueKey
However, I would test how much space you save by not having one.
On Tue, Jul 10, 2012 at 6:27 AM, Sachin Aggarwal <different.sachin@gmail.com
> wrote:
> in my use case i m not deleting any doc from solr i m using batch build on
> data and use solr as filters on data data is very large raw rows are in
> billions and filtered or searched query are in millions...is there any way
> to leave unique key from indexing
>
> On Tue, Jul 10, 2012 at 3:42 PM, Tomás Fernández Löbbe <
> tomasflobbe@gmail.com> wrote:
>
> > In order to support updates (which are treated as deleted + add), the
> > unique key needs to be indexed.
> >
> > Tomás
> >
> > On Tue, Jul 10, 2012 at 6:08 AM, Sachin Aggarwal <
> > different.sachin@gmail.com
> > > wrote:
> >
> > > is it possible not to index and but store the unique key....it will
> make
> > > index size small. i need the unique key to b stored so that i can read
> > the
> > > returned result from the database.
> > >
> > > --
> > >
> > > Thanks & Regards
> > >
> > > Sachin Aggarwal
> > > 7760502772
> > >
> >
>
>
>
> --
>
> Thanks & Regards
>
> Sachin Aggarwal
> 7760502772
>
Re: unique key
Posted by Sachin Aggarwal <di...@gmail.com>.
in my use case i m not deleting any doc from solr i m using batch build on
data and use solr as filters on data data is very large raw rows are in
billions and filtered or searched query are in millions...is there any way
to leave unique key from indexing
On Tue, Jul 10, 2012 at 3:42 PM, Tomás Fernández Löbbe <
tomasflobbe@gmail.com> wrote:
> In order to support updates (which are treated as deleted + add), the
> unique key needs to be indexed.
>
> Tomás
>
> On Tue, Jul 10, 2012 at 6:08 AM, Sachin Aggarwal <
> different.sachin@gmail.com
> > wrote:
>
> > is it possible not to index and but store the unique key....it will make
> > index size small. i need the unique key to b stored so that i can read
> the
> > returned result from the database.
> >
> > --
> >
> > Thanks & Regards
> >
> > Sachin Aggarwal
> > 7760502772
> >
>
--
Thanks & Regards
Sachin Aggarwal
7760502772
Re: unique key
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
In order to support updates (which are treated as deleted + add), the
unique key needs to be indexed.
Tomás
On Tue, Jul 10, 2012 at 6:08 AM, Sachin Aggarwal <different.sachin@gmail.com
> wrote:
> is it possible not to index and but store the unique key....it will make
> index size small. i need the unique key to b stored so that i can read the
> returned result from the database.
>
> --
>
> Thanks & Regards
>
> Sachin Aggarwal
> 7760502772
>