You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Andy Seaborne (Jira)" <ji...@apache.org> on 2021/12/15 12:42:00 UTC
[jira] [Commented] (JENA-2217) tdb2.xloader java.io.IOException: Broken pipe
[ https://issues.apache.org/jira/browse/JENA-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17459906#comment-17459906 ]
Andy Seaborne commented on JENA-2217:
-------------------------------------
Aside:
-Xmx12G
will slow the loader down because it may compete with the sort process.
The java steps do not do operations that consume large amounts of heap.
> tdb2.xloader java.io.IOException: Broken pipe
> ---------------------------------------------
>
> Key: JENA-2217
> URL: https://issues.apache.org/jira/browse/JENA-2217
> Project: Apache Jena
> Issue Type: Bug
> Components: Cmd line tools
> Environment: Rocky linux 8
> openjdk 17.0.1 2021-10-19 LTS
> OpenJDK Runtime Environment 21.9 (build 17.0.1+12-LTS)
> OpenJDK 64-Bit Server VM 21.9 (build 17.0.1+12-LTS, mixed mode, sharing)
> Reporter: Øyvind Gjesdal
> Priority: Major
> Attachments: truthy-import.log
>
>
> When running the tdb2.xloader with
> `
> {code:java}
> JVM_ARGS="-Xmx12G -Xms12G -XX:+UseParallelGC" $JENA_HOME/bin/tdb2.xloader --loc /var/fuseki/databases/wd-truthy --tmpdir /var/fuseki/databases/tmp /var/fuseki/databases/latest-truthy.nt.gz |& tee /home/fuseki/truthy-import.log {code}
> `
>
>
>
> The exception also doesn't exit the application, but continues indefinetly with a new exception, stream closed.
> This is for a snapshot a day prior to 4.3.1.
> [^truthy-import.log]
>
> {code:java}
> java.io.IOException: Broken pipe
> at java.base/java.io.FileOutputStream.writeBytes(Native Method)
> at java.base/java.io.FileOutputStream.write(FileOutputStream.java:349)
> at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
> at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:95)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.hexWrite(ProcNodeTableBuilderX.java:331)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.write(ProcNodeTableBuilderX.java:407)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.node(ProcNodeTableBuilderX.java:396)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.triple(ProcNodeTableBuilderX.java:360)
> at org.apache.jena.riot.system.StreamRDFWrapper.triple(StreamRDFWrapper.java:40)
> at org.apache.jena.system.progress.ProgressStreamRDF.triple(ProgressStreamRDF.java:41)
> at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:55)
> 11:26:56 ERROR Terms :: Sort RC = 2
> at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:43)
> at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTLang.read(RDFParserRegistry.java:186)
> at org.apache.jena.riot.RDFParser.read(RDFParser.java:366)
> at org.apache.jena.riot.RDFParser.parseURI(RDFParser.java:335)
> at org.apache.jena.riot.RDFParser.parse(RDFParser.java:310)
> at org.apache.jena.riot.RDFParserBuilder.parse(RDFParserBuilder.java:552)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$0(ProcNodeTableBuilderX.java:156)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$1(ProcNodeTableBuilderX.java:152)
> at java.base/java.lang.Thread.run(Thread.java:833)
> java.io.IOException: Stream closed
> at java.base/java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:445)
> at java.base/java.io.OutputStream.write(OutputStream.java:162)
> at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
> at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:95)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.hexWrite(ProcNodeTableBuilderX.java:330)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.write(ProcNodeTableBuilderX.java:407)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.node(ProcNodeTableBuilderX.java:394)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX$NodeHashTmpStream.triple(ProcNodeTableBuilderX.java:360)
> at org.apache.jena.riot.system.StreamRDFWrapper.triple(StreamRDFWrapper.java:40)
> at org.apache.jena.system.progress.ProgressStreamRDF.triple(ProgressStreamRDF.java:41)
> at org.apache.jena.riot.lang.LangNTriples.runParser(LangNTriples.java:55)
> at org.apache.jena.riot.lang.LangBase.parse(LangBase.java:43)
> at org.apache.jena.riot.RDFParserRegistry$ReaderRIOTLang.read(RDFParserRegistry.java:186)
> at org.apache.jena.riot.RDFParser.read(RDFParser.java:366)
> at org.apache.jena.riot.RDFParser.parseURI(RDFParser.java:335)
> at org.apache.jena.riot.RDFParser.parse(RDFParser.java:310)
> at org.apache.jena.riot.RDFParserBuilder.parse(RDFParserBuilder.java:552)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$0(ProcNodeTableBuilderX.java:156)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at org.apache.jena.tdb2.xloader.ProcNodeTableBuilderX.lambda$exec$1(ProcNodeTableBuilderX.java:152)
> at java.base/java.lang.Thread.run(Thread.java:833)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)