You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Will Johnson <wj...@GETCONNECTED.COM> on 2007/05/25 21:59:16 UTC
field display values
Has anyone done anything interesting to preserve display values for
field names. Ie my users would like to see
Download Speed (MB/sec): 5
As opposed to:
ds:5
there are options for doing fancy encoding of field names but those seem
less that ideal. What I'd really like to do is at add time:
<add>
<doc>
<field name="foo_dynamic" disp="My Foo">hi</foo>
</doc>
</add>
And then at result time:
<str name="foo_dynamic" disp="My Foo">hi</str>
I've thought of having custom request handlers save this info away and
then add it back in with a customer response writer but this seemed like
it might be a more generally useful type of thing to have.
Thoughts, ideas?
- will
RE: field display values
Posted by Will Johnson <wj...@GETCONNECTED.COM>.
>I can't think of any value add in having Solr keep track of the fact
that
>"ds" means "Download Speed" vs having an external data mapping keep
track
>of that information, since direct access to that info inside of Solr
>wouldn't typically make the performance of requests any faster or
>reduce the size of the responses, it seems like the type of data that
make
>more senese to maintain externally.
I agree that it doesn't make solr requests any faster but it does cut
down on the number of total requests. If solr tells me that for this
doc 'ds' means 'Download Speed' (and it may be different for doc #2)
then I don't have to worry about asking some other database.
>if you can particion your index in this way, then similar metadata docs
>might mke sense for you ... if you can't (becuase every doc turely is
>differnet) then making the "real" documents also store the "metadata"
>about field names can work just as well.
I think what I may end up doing is having a 'fieldmap' field for every
document and store the info there. Then I can pull that out at display
time and rename all the fields accordingly.
Thanks for the suggestions.
- will
RE: field display values
Posted by Chris Hostetter <ho...@fucit.org>.
This would require some storage when the index is built to map between the
internal field name and the "display name" ... since this is not a Lucene
concept it would have to be a higher level concept hat Solr write to disk
directly -- there are currently no concepts like this but that doens't
mean there can't be.
the question becomes: "Is this the type of data that Solr *should* store?"
... in my opinion the answer is no.
I can't think of any value add in having Solr keep track of the fact that
"ds" means "Download Speed" vs having an external data mapping keep track
of that information, since direct access to that info inside of Solr
wouldn't typically make the performance of requests any faster or
reduce the size of the responses, it seems like the type of data that make
more senese to maintain externally.
as to your specific situation...
: I would normally agree but the problem is that I'm making very heavy use
: of the dynamic fields and therefore don't really know what a record
: looks like. Ie the only thing that knows about the data is the input
: data itself. I've added logic to 'solrify' the input field names as
: they come to me in the "Download Speed" format but making the reverse
: happen is impossible from the client side because each record is
: different.
...if every document is truely differnet, then the "ds" field for one doc
may not be the same as the "ds" field for another doc ... which makes it
sound like hte field display names themselves are document specific 'data'
that should be stored as field values.
I have a lot of personal experience with an app (the first Solr app
actually) where the dynamic fields a doc has depend on it's category,
and i actually put the info about the fields (including their display
names and info on how to facet on them) into stored fields special
"metadata documents" which go into the index and then a custom request
handler first says "what category am i interested in?" to find the
relevant metadata doc, and then uses the info found in that doc to both
query the index for the "real" results, as well as to return the "display"
values for all of the important fields.
if you can particion your index in this way, then similar metadata docs
might mke sense for you ... if you can't (becuase every doc turely is
differnet) then making the "real" documents also store the "metadata"
about field names can work just as well.
-Hoss
RE: field display values
Posted by Will Johnson <wj...@GETCONNECTED.COM>.
I would normally agree but the problem is that I'm making very heavy use
of the dynamic fields and therefore don't really know what a record
looks like. Ie the only thing that knows about the data is the input
data itself. I've added logic to 'solrify' the input field names as
they come to me in the "Download Speed" format but making the reverse
happen is impossible from the client side because each record is
different.
- will
-----Original Message-----
From: Ryan McKinley [mailto:ryantxu@gmail.com]
Sent: Friday, May 25, 2007 4:32 PM
To: solr-user@lucene.apache.org
Subject: Re: field display values
Will Johnson wrote:
> Has anyone done anything interesting to preserve display values for
> field names. Ie my users would like to see
>
> Download Speed (MB/sec): 5
>
> As opposed to:
>
> ds:5
>
>
The general model has been to think of solr like SQL... it is only the
database - display choices should be at the client side. It seems easy
enough to have a map on the client with:
"ds" => "Download Speed (MB/sec)"
That said, something like the sql 'as' command would be useful:
SELECT ds as `Download Speed (MB/sec)` FROM table...;
rather then define the field name at index time (as your example
suggests) it makes more sense to define it at query time (or as a
default in the RequestHandler config) Maybe something like:
/select?fl=ds&display.ds=Download Speed (MB/sec)
Maybe this would be a way to specify date formatting?
/select?fl=timestamp&display.timestamp='Year'&display.format.timestamp=Y
YYY
just thoughts...
Re: field display values
Posted by Ryan McKinley <ry...@gmail.com>.
Will Johnson wrote:
> Has anyone done anything interesting to preserve display values for
> field names. Ie my users would like to see
>
> Download Speed (MB/sec): 5
>
> As opposed to:
>
> ds:5
>
>
The general model has been to think of solr like SQL... it is only the
database - display choices should be at the client side. It seems easy
enough to have a map on the client with:
"ds" => "Download Speed (MB/sec)"
That said, something like the sql 'as' command would be useful:
SELECT ds as `Download Speed (MB/sec)` FROM table...;
rather then define the field name at index time (as your example
suggests) it makes more sense to define it at query time (or as a
default in the RequestHandler config) Maybe something like:
/select?fl=ds&display.ds=Download Speed (MB/sec)
Maybe this would be a way to specify date formatting?
/select?fl=timestamp&display.timestamp='Year'&display.format.timestamp=YYYY
just thoughts...