internal field max length?

[Brian Whitman <br...@variogr.am>]

I am sending Solr stored fields of sizes in the 10-50K range. My  
maxFieldLength is 50000, and the field in question is a  
solr.TextField. I am finding that fields that have more than a few K  
of text come back "clipped:" if I try to index the field with 40K of  
text, the search result will show only the *last* 5-10K or so, the  
beginning is missing.

Is there somewhere else I should look for a field trim other than  
maxFieldLength?

Re: internal field max length?

[Ryan McKinley <ry...@gmail.com>]

Looks like it was actually an error with SOLR-133 not handling CDATA
properly.  I fixed it and updated the patch.

at least SOLR-20 ins't to blame!


On 2/21/07, Brian Whitman <br...@variogr.am> wrote:
> On Feb 21, 2007, at 5:10 PM, Yonik Seeley wrote:
> >
> > So far so good for me.
> > I started with example/exampledocs/solr.xml and added an additional
> > field value for "features" of size 500K
> > It starts with "this is the first line", then repeats the ASL over and
> > over, then
> > ends with "this is the last line".
> >
> > I posted via post.sh (curl), and then retrieved by searching for the
> > id "solr", and
> > observed the complete field returned.
>
>
> I just did the same thing as you.. with the same results. It must be
> SOLR-20 or some brain dead thing I'm doing (I suspect the latter, but
> we'll see.)
>
> -Brian
>
>
>
>
>

Re: internal field max length?

[Brian Whitman <br...@variogr.am>]

On Feb 21, 2007, at 5:10 PM, Yonik Seeley wrote:
>
> So far so good for me.
> I started with example/exampledocs/solr.xml and added an additional
> field value for "features" of size 500K
> It starts with "this is the first line", then repeats the ASL over and
> over, then
> ends with "this is the last line".
>
> I posted via post.sh (curl), and then retrieved by searching for the
> id "solr", and
> observed the complete field returned.


I just did the same thing as you.. with the same results. It must be  
SOLR-20 or some brain dead thing I'm doing (I suspect the latter, but  
we'll see.)

-Brian

Re: internal field max length?

[Yonik Seeley <yo...@apache.org>]

On 2/21/07, Brian Whitman <br...@variogr.am> wrote:
> > Ouch... sounds serious (assuming you aren't talking about
> > highlighting).
> > Could you open a JIRA issue and describe or attach a test that can
> > reproduce it?
> > I'll try to reproduce this myself in the meantime.

So far so good for me.
I started with example/exampledocs/solr.xml and added an additional
field value for "features" of size 500K
It starts with "this is the first line", then repeats the ASL over and
over, then
ends with "this is the last line".

I posted via post.sh (curl), and then retrieved by searching for the
id "solr", and
observed the complete field returned.


> Not highlighting, no. I'll try to make a test case. I am using the
> SOLR-20 client to post the data, so there's still a chance that's the
> culprit. I will try with straight HTTP.

please do... that might be it.

-Yonik

Re: internal field max length?

[Brian Whitman <br...@variogr.am>]

> Ouch... sounds serious (assuming you aren't talking about  
> highlighting).
> Could you open a JIRA issue and describe or attach a test that can  
> reproduce it?
> I'll try to reproduce this myself in the meantime.


Not highlighting, no. I'll try to make a test case. I am using the  
SOLR-20 client to post the data, so there's still a chance that's the  
culprit. I will try with straight HTTP.

-Brian

Re: internal field max length?

[Yonik Seeley <yo...@apache.org>]

On 2/21/07, Brian Whitman <br...@variogr.am> wrote:
> I am sending Solr stored fields of sizes in the 10-50K range. My
> maxFieldLength is 50000, and the field in question is a
> solr.TextField. I am finding that fields that have more than a few K
> of text come back "clipped:" if I try to index the field with 40K of
> text, the search result will show only the *last* 5-10K or so, the
> beginning is missing.
>
> Is there somewhere else I should look for a field trim other than
> maxFieldLength?

Ouch... sounds serious (assuming you aren't talking about highlighting).
Could you open a JIRA issue and describe or attach a test that can reproduce it?
I'll try to reproduce this myself in the meantime.

-Yonik