Segments with 500KB size (source: "flush") in a cluster with ramBufferSize of 2GB

[Daniel Ortega <da...@gmail.com>]

Hi Guys,

Could someone explain me why I have segments of 500 KB (with source "flush"
and only with 91 documents) if I have a ramBufferSizeMB of 2GB
and maxBufferedDocs not definined?

Thanks in advance,
Daniel

Re: Segments with 500KB size (source: "flush") in a cluster with ramBufferSize of 2GB

[Daniel Ortega <da...@gmail.com>]

Hi Erick,

1) maxDocs setting is not defined in our config. As far as we know there
isn't a default value.

2) ramBufferSizeMB cannot be exceeded because written segment size are less
than 1 MB and we have defined a maximum of 2 GB.

3) The autocommit interval is 15 minutes. This could be the reason but it's
weird because we are updating more than one thousand documents per second
(across the cluster).

4) We disabled explicit commits from clients :)

5) We're using SolrJ but with commitWithin option not defined.

Anyway thanks for your reply!

2017-07-06 16:56 GMT+02:00 Erick Erickson <er...@gmail.com>:

> You'll get a new segment whenever your time expires as well. Segments
> are created whenever the _first_ of
> 1> maxDocs exceeded
> 2> ramBufferSizeMB is exceded
> 3> the autocommit _time_ interval is exceeded
> 4> someone does a commit
> 5> if someone is indexing via SolrJ and specifies a "commitWithin" for
> one of the calls.
>
> Can you guarantee that none of these occur?
>
> Best,
> Erick
>
> On Thu, Jul 6, 2017 at 6:14 AM, Daniel Ortega
> <da...@gmail.com> wrote:
> > Hi Guys,
> >
> > Could someone explain me why I have segments of 500 KB (with source
> "flush"
> > and only with 91 documents) if I have a ramBufferSizeMB of 2GB
> > and maxBufferedDocs not definined?
> >
> > Thanks in advance,
> > Daniel
>

Re: Segments with 500KB size (source: "flush") in a cluster with ramBufferSize of 2GB

[Erick Erickson <er...@gmail.com>]

You'll get a new segment whenever your time expires as well. Segments
are created whenever the _first_ of
1> maxDocs exceeded
2> ramBufferSizeMB is exceded
3> the autocommit _time_ interval is exceeded
4> someone does a commit
5> if someone is indexing via SolrJ and specifies a "commitWithin" for
one of the calls.

Can you guarantee that none of these occur?

Best,
Erick

On Thu, Jul 6, 2017 at 6:14 AM, Daniel Ortega
<da...@gmail.com> wrote:
> Hi Guys,
>
> Could someone explain me why I have segments of 500 KB (with source "flush"
> and only with 91 documents) if I have a ramBufferSizeMB of 2GB
> and maxBufferedDocs not definined?
>
> Thanks in advance,
> Daniel