Re: Closed connection issue while doing dataimport

[santosh sidnal <si...@gmail.com>]

Thanks Shwan, that was good point to consider but we had porblem with
holdability param in data-config.xml and Oracle 12c version of DB and
client.

BY remove Holdability parameter we are able to solve the issue.

On 28 March 2017 at 18:19, Shawn Heisey <ap...@elyograg.org> wrote:

> On 3/27/2017 7:13 PM, santosh sidnal wrote:
> > i am facing closed connection issue while doing dataimporter, any
> solution
> > to this> stack trace is as below
> >
> >
> > [3/27/17 8:54:41:399 CDT] 000000b4 OracleDataSto >  findMappingClass for
> :
> > Entry
> >                                  java.sql.SQLRecoverableException:
> Closed
> > Connection
>
> Does the import appear to work correctly at first, then stop before it's
> done with this exception after a few hours?  If so, then I think you may
> be running into a situation where the merge scheduler has multiple
> merges scheduled and stops the incoming indexing thread until the
> largest merge is done.  If the indexing thread is stopped for long
> enough, JDBC will disconnect the database, and when the indexing thread
> finally starts back up, the dataimporter finds that it cannot read from
> the database any more.
>
> If this is what's happening, then the solution is to allow the merge
> scheduler to schedule more merges simultaneously.  Here's the
> indexConfig that I use in solrconfig.xml:
>
> <indexConfig>
>   <mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
>     <int name="maxMergeAtOnce">35</int>
>     <int name="segmentsPerTier">35</int>
>     <int name="maxMergeAtOnceExplicit">105</int>
>   </mergePolicy>
>   <mergeScheduler class="org.apache.lucene.index.
> ConcurrentMergeScheduler">
>     <int name="maxThreadCount">1</int>
>     <int name="maxMergeCount">6</int>
>   </mergeScheduler>
> </indexConfig>
>
> You don't need the mergePolicy part of this config to solve this issue.
> That is the settings that I use, but this greatly increases the number
> of files that can be in an index, which may require OS tuning for max
> open files.
>
> The mergeScheduler is the important part, and specifically
> maxMergeCount.  These settings are for standard spinning-magnetic
> disks.  I increase the maxMergeCount to 6, so more merges can be
> scheduled without halting the indexing thread.  The maxThreadCount value
> should be set to 1 if you're using standard disks.  If you're using SSD,
> then you can bump it a little bit, because SSD can easily handle
> multiple threads randomly writing to the disk.  I don't know what values
> are appropriate for SSD, but I would probably start with 2-4.
>
> Thanks,
> Shawn
>
>


-- 
Regards,
Santosh Sidnal