[jira] [Commented] (NIFI-6777) Data Provenance SIGSEGV : Search Lucene Index-1

["Firenz (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989925#comment-16989925 ] 

Firenz commented on NIFI-6777:
------------------------------

The disabling of G1GC did not solved our issue. We've just have a "double crash" : Node 1 & 3 of our 3-nodes cluster.
 * Would using OpenJDK change somehting ? 
 * Would upgrading Nifi 1.8.0 to OpenJDK 9/11/13 is possible ?

> Data Provenance SIGSEGV : Search Lucene Index-1
> -----------------------------------------------
>
>                 Key: NIFI-6777
>                 URL: https://issues.apache.org/jira/browse/NIFI-6777
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.8.0
>         Environment: Linux CentOS Linux release 7.5.1804 (Core)
> Linux 3.10.0-862.11.6.el7.x86_64
> jre1.8.0_181-amd64 et jre1.8.0_221-amd64
>            Reporter: Firenz
>            Priority: Major
>
> NIFI crashes with a core dump when using data provenance (5 times in 1 months, 4 differents VMs).
>  
> It happens radomly when one of my data analyst is using dataprovenance, but the thread is always "Search Lucene Index-1". My monitoring does not show any memory or thread issues (overload...)
>  
> Any advice ? (I cannot attach the hs_err_pid54305.log, token issue with Jira ?)
>  
> Beginning of core dump : 
> {code:java}
> # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build 1.8.0_181-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # J 33233 C2 org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock()V (448 bytes) @ 0x00007f9c52baabf9 [0x00007f9
> c52baaa40+0x1b9]
> #
> # Core dump written. Default location: /appli/nifi-1.8.0/core or core.54305
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #---------------  T H R E A D  ---------------Current thread (0x00007f9c3c02b000):  JavaThread "Search Lucene Index-1" daemon [_thread_in_Java, id=64766, stack(0x00007f9b9a
> c4d000,0x00007f9b9ad4e000)]siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007f9b82019d0c
> {code}
> My Dataprovenance config :
> {code:java}
> # Persistent Provenance Repository Properties
> nifi.provenance.repository.directory.default=/data_nifi_provenance_repo/1.8.0
> nifi.provenance.repository.max.storage.time=480 hours
> nifi.provenance.repository.max.storage.size=9 GB
> nifi.provenance.repository.rollover.time=30 secs
> nifi.provenance.repository.rollover.size=100 MB
> nifi.provenance.repository.query.threads=2
> nifi.provenance.repository.index.threads=2
> nifi.provenance.repository.compress.on.rollover=true
> nifi.provenance.repository.always.sync=false
> nifi.provenance.repository.journal.count=16
> # Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
> # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
> nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
> # FlowFile Attributes that should be indexed and made searchable.  Some examples to consider are filename, uuid, mime.type
> nifi.provenance.repository.indexed.attributes=external_id, batch_id, property_id, property_code, http.headers.x-request-id
> # Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
> # but should provide better performance
> nifi.provenance.repository.index.shard.size=500 MB
> # Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
> # the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
> nifi.provenance.repository.max.attribute.length=65536
> nifi.provenance.repository.concurrent.merge.threads=2
> nifi.provenance.repository.warm.cache.frequency=1 hour
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)