You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Øyvind Gjesdal (Jira)" <ji...@apache.org> on 2021/12/22 16:27:00 UTC
[jira] [Comment Edited] (JENA-2217) tdb2.xloader java.io.IOException: Broken pipe
[ https://issues.apache.org/jira/browse/JENA-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463943#comment-17463943 ]
Øyvind Gjesdal edited comment on JENA-2217 at 12/22/21, 4:26 PM:
-----------------------------------------------------------------
This was my mistake:
Having a tmp dir which doesn't exist, causes sort to fail in the exec, when memory is running out and it then tries to use the tmp dir. I had deleted it between runs, and expected it to be created.
Maybe it could fail earlier, or try to create the directory?
To debug, I added to ProcNodeTableBuilderX, which gave me a usable error message.
```
@@ -226,13 +227,14 @@ public class ProcNodeTableBuilderX {
try
{ int x = proc2.waitFor(); if ( x != 0 ) - FmtLog.error(LOG2, "Sort RC = %d", x); + FmtLog.error(LOG2, "Sort RC = %d, Error message: %s", x, new String(proc2.getErrorStream().readAllBytes())); else LOG2.info("Sort finished"); - }
catch (InterruptedException e)
{ + }
catch (InterruptedException|IOException e) {
```
was (Author: oyvind.gjesdal):
This was my mistake:
Having a tmp dir which doesn't exist, causes sort to fail in the exec, when memory is running out and it then tries to use the tmp dir. I had deleted it between runs, and expected it to be created.
Maybe it could fail earlier, or try to create the directory?
To debug, so I added to ProcNodeTableBuilderX, which gave me a usable error message.
```
@@ -226,13 +227,14 @@ public class ProcNodeTableBuilderX {
try {
int x = proc2.waitFor();
if ( x != 0 )
- FmtLog.error(LOG2, "Sort RC = %d", x);
+ FmtLog.error(LOG2, "Sort RC = %d, Error message: %s", x, new String(proc2.getErrorStream().readAllBytes()));
else
LOG2.info("Sort finished");
- } catch (InterruptedException e) {
+ } catch (InterruptedException|IOException e) {
```
> tdb2.xloader java.io.IOException: Broken pipe
> ---------------------------------------------
>
> Key: JENA-2217
> URL: https://issues.apache.org/jira/browse/JENA-2217
> Project: Apache Jena
> Issue Type: Bug
> Components: Cmd line tools
> Environment: Rocky linux 8
> openjdk 17.0.1 2021-10-19 LTS
> OpenJDK Runtime Environment 21.9 (build 17.0.1+12-LTS)
> OpenJDK 64-Bit Server VM 21.9 (build 17.0.1+12-LTS, mixed mode, sharing)
> Reporter: Øyvind Gjesdal
> Assignee: Andy Seaborne
> Priority: Major
> Attachments: truthy-import.log
>
>
> When running the tdb2.xloader with
> `
> {code:java}
> JVM_ARGS="-Xmx12G -Xms12G -XX:+UseParallelGC" $JENA_HOME/bin/tdb2.xloader --loc /var/fuseki/databases/wd-truthy --tmpdir /var/fuseki/databases/tmp /var/fuseki/databases/latest-truthy.nt.gz |& tee /home/fuseki/truthy-import.log {code}
> `
>
>
>
> The exception also doesn't exit the application, but continues indefinetly with a new exception, stream closed.
> This is for a snapshot a day prior to 4.3.1.
> [^truthy-import.log]
>
> {code:java}
> java.io.IOException: Broken pipe
> at java.base/java.io.FileOutputStream.writeBytes(Native Method)
> at java.base/java.io.FileOutputStream.write(FileOutputStream.java:349)
> at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
> at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:95)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.hexWrite(ProcNodeTableBuilderX.java:331)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.write(ProcNodeTableBuilderX.java:407)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.node(ProcNodeTableBuilderX.java:396)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.triple(ProcNodeTableBuilderX.java:360)
> at org.apache.jena.riot.system.StreamRDFWrapper.triple(StreamRDFWrapper.java:40)
> at org.apache.jena.system.progress.ProgressStreamRDF.triple(ProgressStreamRDF.java:41)
> at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:55)
> 11:26:56 ERROR Terms :: Sort RC = 2
> at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:43)
> at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTLang.read(RDFParserRegistry.java:186)
> at org.apache.jena.riot.RDFParser.read(RDFParser.java:366)
> at org.apache.jena.riot.RDFParser.parseURI(RDFParser.java:335)
> at org.apache.jena.riot.RDFParser.parse(RDFParser.java:310)
> at org.apache.jena.riot.RDFParserBuilder.parse(RDFParserBuilder.java:552)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$0(ProcNodeTableBuilderX.java:156)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$1(ProcNodeTableBuilderX.java:152)
> at java.base/java.lang.Thread.run(Thread.java:833)
> java.io.IOException: Stream closed
> at java.base/java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:445)
> at java.base/java.io.OutputStream.write(OutputStream.java:162)
> at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
> at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:95)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.hexWrite(ProcNodeTableBuilderX.java:330)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.write(ProcNodeTableBuilderX.java:407)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.node(ProcNodeTableBuilderX.java:394)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.triple(ProcNodeTableBuilderX.java:360)
> at org.apache.jena.riot.system.StreamRDFWrapper.triple(StreamRDFWrapper.java:40)
> at org.apache.jena.system.progress.ProgressStreamRDF.triple(ProgressStreamRDF.java:41)
> at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:55)
> at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:43)
> at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTLang.read(RDFParserRegistry.java:186)
> at org.apache.jena.riot.RDFParser.read(RDFParser.java:366)
> at org.apache.jena.riot.RDFParser.parseURI(RDFParser.java:335)
> at org.apache.jena.riot.RDFParser.parse(RDFParser.java:310)
> at org.apache.jena.riot.RDFParserBuilder.parse(RDFParserBuilder.java:552)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$0(ProcNodeTableBuilderX.java:156)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$1(ProcNodeTableBuilderX.java:152)
> at java.base/java.lang.Thread.run(Thread.java:833)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)