You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drat.apache.org by Chris Mattmann <ma...@apache.org> on 2017/11/16 23:30:57 UTC
SIGSEV from Lucene upgrades in OODT....
Hey Tom,
I’ve noticed while testing in Apache DRAT that the Lucene upgrades you did
for the file manager regularly on the drat-vm Ubuntu box cause SIGSEV failures
like the following:
WARNING: No Metadata specified for product [TestMetReader.java] for required field [DataVersion]: Attempting to continue processing metadata
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f5afcc030f7, pid=30823, tid=0x00007f5ab9d4f700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build 1.8.0_151-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# J 1360 C2 org.apache.lucene.store.ByteBufferIndexInput.readByte()B (126 bytes) @ 0x00007f5afcc030f7 [0x00007f5afcc030a0+0x57]
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/mattmann/drat/deploy/filemgr/bin/hs_err_pid30823.log
Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
WARNING: No Metadata specified for product [TestMetReader.java] for required field [DataVersion]: Attempting to continue processing metadata
Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [GenericFile]
Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [GenericFile]
Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
INFO: Runn
Steps to reproduce:
1. Clone and install Apache DRAT 1.0-SNAPSHOT from apache/drat:
2. Run DRAT on Apache OODT github https://github.com/apache/oodt.git or any repo with thousands of files using Proteus
3. Watch an error like the above appear consistently about half-way through crawling…
There are some links online and this open LUCENE issue:
https://issues.apache.org/jira/browse/LUCENE-6943
Tom can you help investigate?
Cheers,
Chris
Re: SIGSEV from Lucene upgrades in OODT....
Posted by Chris Mattmann <ma...@apache.org>.
Hey Lewis,
Agreed here is the info (BTW I believe you should be able to access drat-vm.a.o):
javmattmann@drat-vm:~$ java -version
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
mattmann@drat-vm:~$
Let me know what you find Lewis….check the steps below.
Cheers,
Chris
On 11/16/17, 6:01 PM, "lewis john mcgibbney" <le...@apache.org> wrote:
Which version of Java is running on drat-vm Chris?
Can you post a link to the DRAT Solr schema being used?
It would be nice to address this before releasing current OODT RC. Not a
total requirement but would be good to get to the bottom of.
Lewis
On Thu, Nov 16, 2017 at 3:36 PM Tom Barber <to...@meteorite.bi> wrote:
> Hrmmmm, I shall take a look!
>
> On 16/11/17 23:30, Chris Mattmann wrote:
> > Hey Tom,
> >
> >
> >
> > I’ve noticed while testing in Apache DRAT that the Lucene upgrades you
> did
> > for the file manager regularly on the drat-vm Ubuntu box cause SIGSEV
> failures
> > like the following:
> >
> >
> >
> > WARNING: No Metadata specified for product [TestMetReader.java] for
> required field [DataVersion]: Attempting to continue processing metadata
> >
> > #
> >
> > # A fatal error has been detected by the Java Runtime Environment:
> >
> > #
> >
> > # SIGSEGV (0xb) at pc=0x00007f5afcc030f7, pid=30823,
> tid=0x00007f5ab9d4f700
> >
> > #
> >
> > # JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build
> 1.8.0_151-b12)
> >
> > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode
> linux-amd64 compressed oops)
> >
> > # Problematic frame:
> >
> > # J 1360 C2 org.apache.lucene.store.ByteBufferIndexInput.readByte()B
> (126 bytes) @ 0x00007f5afcc030f7 [0x00007f5afcc030a0+0x57]
> >
> > #
> >
> > # Failed to write core dump. Core dumps have been disabled. To enable
> core dumping, try "ulimit -c unlimited" before starting Java again
> >
> > #
> >
> > # An error report file with more information is saved as:
> >
> > # /home/mattmann/drat/deploy/filemgr/bin/hs_err_pid30823.log
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
> >
> > WARNING: No Metadata specified for product [TestMetReader.java] for
> required field [DataVersion]: Attempting to continue processing metadata
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> >
> > INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
> product type: [GenericFile]
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> >
> > INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
> for product type: [GenericFile]
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> >
> > INFO: Runn
> >
> >
> >
> >
> >
> > Steps to reproduce:
> >
> >
> >
> > 1. Clone and install Apache DRAT 1.0-SNAPSHOT from apache/drat:
> >
> > 2. Run DRAT on Apache OODT github
> https://github.com/apache/oodt.git or any repo with thousands of files
> using Proteus
> >
> > 3. Watch an error like the above appear consistently about
> half-way through crawling…
> >
> >
> >
> > There are some links online and this open LUCENE issue:
> >
> >
> >
> > https://issues.apache.org/jira/browse/LUCENE-6943
> >
> >
> >
> > Tom can you help investigate?
> >
> >
> >
> > Cheers,
> >
> > Chris
> >
> >
> >
> >
> >
> >
>
> --
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
Re: SIGSEV from Lucene upgrades in OODT....
Posted by lewis john mcgibbney <le...@apache.org>.
Which version of Java is running on drat-vm Chris?
Can you post a link to the DRAT Solr schema being used?
It would be nice to address this before releasing current OODT RC. Not a
total requirement but would be good to get to the bottom of.
Lewis
On Thu, Nov 16, 2017 at 3:36 PM Tom Barber <to...@meteorite.bi> wrote:
> Hrmmmm, I shall take a look!
>
> On 16/11/17 23:30, Chris Mattmann wrote:
> > Hey Tom,
> >
> >
> >
> > I’ve noticed while testing in Apache DRAT that the Lucene upgrades you
> did
> > for the file manager regularly on the drat-vm Ubuntu box cause SIGSEV
> failures
> > like the following:
> >
> >
> >
> > WARNING: No Metadata specified for product [TestMetReader.java] for
> required field [DataVersion]: Attempting to continue processing metadata
> >
> > #
> >
> > # A fatal error has been detected by the Java Runtime Environment:
> >
> > #
> >
> > # SIGSEGV (0xb) at pc=0x00007f5afcc030f7, pid=30823,
> tid=0x00007f5ab9d4f700
> >
> > #
> >
> > # JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build
> 1.8.0_151-b12)
> >
> > # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode
> linux-amd64 compressed oops)
> >
> > # Problematic frame:
> >
> > # J 1360 C2 org.apache.lucene.store.ByteBufferIndexInput.readByte()B
> (126 bytes) @ 0x00007f5afcc030f7 [0x00007f5afcc030a0+0x57]
> >
> > #
> >
> > # Failed to write core dump. Core dumps have been disabled. To enable
> core dumping, try "ulimit -c unlimited" before starting Java again
> >
> > #
> >
> > # An error report file with more information is saved as:
> >
> > # /home/mattmann/drat/deploy/filemgr/bin/hs_err_pid30823.log
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
> >
> > WARNING: No Metadata specified for product [TestMetReader.java] for
> required field [DataVersion]: Attempting to continue processing metadata
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> >
> > INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
> product type: [GenericFile]
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> >
> > INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
> for product type: [GenericFile]
> >
> > Nov 16, 2017 11:25:40 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> >
> > INFO: Runn
> >
> >
> >
> >
> >
> > Steps to reproduce:
> >
> >
> >
> > 1. Clone and install Apache DRAT 1.0-SNAPSHOT from apache/drat:
> >
> > 2. Run DRAT on Apache OODT github
> https://github.com/apache/oodt.git or any repo with thousands of files
> using Proteus
> >
> > 3. Watch an error like the above appear consistently about
> half-way through crawling…
> >
> >
> >
> > There are some links online and this open LUCENE issue:
> >
> >
> >
> > https://issues.apache.org/jira/browse/LUCENE-6943
> >
> >
> >
> > Tom can you help investigate?
> >
> >
> >
> > Cheers,
> >
> > Chris
> >
> >
> >
> >
> >
> >
>
> --
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
Re: SIGSEV from Lucene upgrades in OODT....
Posted by Tom Barber <to...@meteorite.bi>.
Hrmmmm, I shall take a look!
On 16/11/17 23:30, Chris Mattmann wrote:
> Hey Tom,
>
>
>
> I’ve noticed while testing in Apache DRAT that the Lucene upgrades you did
> for the file manager regularly on the drat-vm Ubuntu box cause SIGSEV failures
> like the following:
>
>
>
> WARNING: No Metadata specified for product [TestMetReader.java] for required field [DataVersion]: Attempting to continue processing metadata
>
> #
>
> # A fatal error has been detected by the Java Runtime Environment:
>
> #
>
> # SIGSEGV (0xb) at pc=0x00007f5afcc030f7, pid=30823, tid=0x00007f5ab9d4f700
>
> #
>
> # JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build 1.8.0_151-b12)
>
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode linux-amd64 compressed oops)
>
> # Problematic frame:
>
> # J 1360 C2 org.apache.lucene.store.ByteBufferIndexInput.readByte()B (126 bytes) @ 0x00007f5afcc030f7 [0x00007f5afcc030a0+0x57]
>
> #
>
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>
> #
>
> # An error report file with more information is saved as:
>
> # /home/mattmann/drat/deploy/filemgr/bin/hs_err_pid30823.log
>
> Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
>
> WARNING: No Metadata specified for product [TestMetReader.java] for required field [DataVersion]: Attempting to continue processing metadata
>
> Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [GenericFile]
>
> Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [GenericFile]
>
> Nov 16, 2017 11:25:40 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>
> INFO: Runn
>
>
>
>
>
> Steps to reproduce:
>
>
>
> 1. Clone and install Apache DRAT 1.0-SNAPSHOT from apache/drat:
>
> 2. Run DRAT on Apache OODT github https://github.com/apache/oodt.git or any repo with thousands of files using Proteus
>
> 3. Watch an error like the above appear consistently about half-way through crawling…
>
>
>
> There are some links online and this open LUCENE issue:
>
>
>
> https://issues.apache.org/jira/browse/LUCENE-6943
>
>
>
> Tom can you help investigate?
>
>
>
> Cheers,
>
> Chris
>
>
>
>
>
>