You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Andy Seaborne (JIRA)" <ji...@apache.org> on 2015/07/23 15:08:05 UTC
[jira] [Comment Edited] (JENA-977) tdbloader2 script refactoring
[ https://issues.apache.org/jira/browse/JENA-977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638765#comment-14638765 ]
Andy Seaborne edited comment on JENA-977 at 7/23/15 1:07 PM:
-------------------------------------------------------------
D.ttl contains 1 triple.
Case 1: empty database directory DB exists
{noformat}
$ bin/tdbloader2 --loc DB ~/tmp/D.ttl
13:56:43 INFO -- TDB Bulk Loader Start
13:56:43 INFO Data Load Phase
13:56:43 INFO Got 1 data files to load
13:56:43 INFO Data file 1: /home/afs/tmp/D.ttl
INFO Load: /home/afs/tmp/D.ttl -- 2015/07/23 13:56:44 BST
INFO Total: 1 tuples : 0.10 seconds : 10.00 tuples/sec [2015/07/23 13:56:44 BST]
13:56:44 INFO Data Load Phase Completed
13:56:44 INFO Index Building Phase
13:56:44 INFO Creating Index SPO
df: '/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/DB//SPO-txt': No such file or directory
/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/bin/tdbloader2common: line 71: 100 - : syntax error: operand expected (error token is "- ")
/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/bin/tdbloader2index: line 331: [: : integer expression expected
/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/bin/tdbloader2index: line 342: [: total used free shared buffers cached
Mem: 33687789568 8906674176 24781115392 111333376 454799360 4349825024
-/+ buffers/cache: 4102049792 29585739776
Swap: 34359734272 0 34359734272: integer expression expected
13:56:44 WARN Unable to determine free memory on your OS, can't check whether sort will be in-memory or external sort using Temp Directory /tmp/
13:56:44 INFO Sort SPO
{noformat}
and it hangs at that point.
Case 2: database directory DB does not exist
{noformat}
$ bin/tdbloader2 --loc DB ~/tmp/D.ttl
13:59:44 INFO -- TDB Bulk Loader Start
find: ‘/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/DB’: No such file or directory
13:59:44 INFO Data Load Phase
... as before ...
{noformat}
Also: There are different formats for the logging.
was (Author: andy.seaborne):
Case 1: empty database directory DB exists
{noformat}
$ bin/tdbloader2 --loc DB ~/tmp/D.ttl
13:56:43 INFO -- TDB Bulk Loader Start
13:56:43 INFO Data Load Phase
13:56:43 INFO Got 1 data files to load
13:56:43 INFO Data file 1: /home/afs/tmp/D.ttl
INFO Load: /home/afs/tmp/D.ttl -- 2015/07/23 13:56:44 BST
INFO Total: 1 tuples : 0.10 seconds : 10.00 tuples/sec [2015/07/23 13:56:44 BST]
13:56:44 INFO Data Load Phase Completed
13:56:44 INFO Index Building Phase
13:56:44 INFO Creating Index SPO
df: '/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/DB//SPO-txt': No such file or directory
/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/bin/tdbloader2common: line 71: 100 - : syntax error: operand expected (error token is "- ")
/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/bin/tdbloader2index: line 331: [: : integer expression expected
/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/bin/tdbloader2index: line 342: [: total used free shared buffers cached
Mem: 33687789568 8906674176 24781115392 111333376 454799360 4349825024
-/+ buffers/cache: 4102049792 29585739776
Swap: 34359734272 0 34359734272: integer expression expected
13:56:44 WARN Unable to determine free memory on your OS, can't check whether sort will be in-memory or external sort using Temp Directory /tmp/
13:56:44 INFO Sort SPO
{noformat}
and it hangs at that point.
Case 2: database directory DB does not exist
{noformat}
$ bin/tdbloader2 --loc DB ~/tmp/D.ttl
13:59:44 INFO -- TDB Bulk Loader Start
find: ‘/home/afs/Release/apache-jena-3.0.0-SNAPSHOT/DB’: No such file or directory
13:59:44 INFO Data Load Phase
... as before ...
{noformat}
Also: There are different formats for the logging.
> tdbloader2 script refactoring
> -----------------------------
>
> Key: JENA-977
> URL: https://issues.apache.org/jira/browse/JENA-977
> Project: Apache Jena
> Issue Type: Improvement
> Components: TDB
> Affects Versions: Jena 2.13.0
> Reporter: Rob Vesse
> Assignee: Rob Vesse
> Priority: Blocker
> Fix For: Jena 2.13.1, Jena 3.0.0
>
>
> As noted on the dev list the current scripts are a little rough around the edges, work items include:
> - Splitting data and index phase into separate scripts
> - Being able to restart a build from a later phase
> - Progress monitoring for the sort portion of indexing
> - Warning if sort is using a disk where you may have insufficient space
> - Better usage summaries
> - Better argument handling (avoid relying on magic environment variables wherever possible)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)