You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stanbol.apache.org by Antero Duarte <a....@gmail.com> on 2016/04/05 13:59:21 UTC
Indexing generic rdf: yago
Hello there,
I have been struggling with building indexes from generic rdf and even
using default configuration for more popular sources like dbpedia.
I found an indexing tool online configured to index yago, at
https://github.com/ChalithaUdara/Stanbol-Yago-Site.
Everything seemed to be going well until it got into this loop:
11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'condition' valid , namespace '
http://www.kinjal.com/condition:' invalid -> mapping ignored!
11:17:26,576 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
12:17:26,856 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
http://prefix.cc/nsogi:' invalid -> mapping ignored!
12:17:26,918 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'category' valid , namespace '
http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
12:17:26,950 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
http://bio2rdf.org/chebi:' invalid -> mapping ignored!
12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'call' valid , namespace '
http://webofcode.org/wfn/call:' invalid -> mapping ignored!
12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
http://bg.dbpedia.org/resource/?????????:' invalid -> mapping ignored!
12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'condition' valid , namespace '
http://www.kinjal.com/condition:' invalid -> mapping ignored!
12:17:27,042 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
It happened to me before with the dbpedia index and at first I thought it
was some problem with the rdf source, and since theses messages are logged
at WARN level, I simply ignored them. but after days, the indexing/tdb
directory stayed the same size even though there are still files in the
indexing/resources/rdfdata directory. Then I realised that these messages
follow a pattern and they are logged every hour with precision to the
second, which seems weird. Also, they are always the same messages. This
led me to think that the indexing tool is stuck in a loop and that's why it
is not moving any further. I think it is important to say that the one hour
time span between messages is the same for the dbpedia index and for the
yago index, the yago index is much bigger.
I have been constantly running `watch du * -s` in the resources directory
for days to check for size changes and nothing is changing and hasn't
changed for days.
I don't know if this is some problem with the configuration, but since I
didn't configure it myself, I assumed that what I got from github would be
a working configuration for this specific index.
I have a few questions related to this problem:
1) Is it safe to cancel the indexing tool and start again without changing
what's in the rdfdata and imported directories? Could this help at all?
2) What can possibly be causing this problem?
3) Why is it looping and logging every hour (accurate to the second)?
If there is any extra information I can provide that would help
understanding what the problem is here, tell me what it is and I will
provide it.
Regards,
Antero
Re: Indexing generic rdf: yago
Posted by "A. Soroka" <aj...@virginia.edu>.
1) tdbstats outputs, as the name hints at, statistics about the database. It's not a dump. If you go and look at the file it gave you, you will see that it talks about things like "how many triples have a certain predicate" and the like. TDB can use this kind of info to optimize queries:
https://jena.apache.org/documentation/tdb/optimizer.html#statistics-rule-file
There is a utility that dumps tuples: tdbdump:
https://jena.apache.org/documentation/tdb/commands.html#tdbdump
2) tdbstats doesn't do anything to the database on which it is run.
3) It's possible that the Stanbol indexer code is using the from-scratch TDB bulkloader. The reason that might be relevant is because that loader uses sorting to sort tuples before loading them, in order to use fast loading methods. That is a step that could present the kind of behavior you saw, but it doesn't sounds that likely. The database shouldn't have been building up before then.
If it's running now, after it finishes you can do a post-mortem, right?
---
A. Soroka
The University of Virginia Library
> On Apr 6, 2016, at 3:53 PM, Antero Duarte <a....@gmail.com> wrote:
>
> Hi,
>
> Right, as an update to my last message, I changed the heap size for
> tdbstats and redirected the output to a file which has 1,005,822 lines
> (output of "wc -l"). Now if this directly relates to the number of
> entities, I don't think it finished because the initial size of the files
> in rdfdata/ was around 70GB.
>
> One interesting thing though, the first time I ran tdbstats, the size of
> the tdb directory increased... Not much and only once, it has been stable
> ever since. Because I never used tdbstats before, I don't know if this is
> the indexing tool showing it's still running or just tdbstats itself
> changing something in the directory.
>
> I restarted the indexing tool and the size of tdb seems to be increasing
> again and the tool seems to be working well... Could it be that I just
> needed to clear whatever was in RAM because it is a huge index?
>
> I will see how it goes from now on and if further problems arise I will ask
> again.
>
> Thank you for your help.
> Antero
>
> On Wed, 6 Apr 2016 at 20:04 A. Soroka <aj...@virginia.edu> wrote:
>
>> TDB command-line tools should respond to ordinary usage of the JAVA_OPTS
>> environment variable, so you should be able to do something like:
>>
>> JAVA_OPTS="-Xmx2G" tdbstats --loc=./tdb
>>
>> to run with a 2GB heap, for example.
>>
>> One thing to note is that if you were even able to access the TDB database
>> at all (which it seems you were because your attempt to use tdbstats didn't
>> error out instantly) it means that no other process was using the database,
>> particularly that the indexer was not. Only one system process can use a
>> TDB database at a time. Now, that may or may not mean that the indexer was
>> actually done doing stuff with the database: I don't know enough about how
>> it works to know that, although the Stanbol devs on here would. It just
>> means that at the moment you tried to run tdbstats, the indexer had
>> relinquished control of the TDB database, at least for the moment.
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>> On Apr 5, 2016, at 6:42 PM, Antero Duarte <a....@gmail.com> wrote:
>>>
>>> Fair enough... I just wanted to figure out if it is actually stopped
>> before
>>> stopping it, but I will do that tomorrow.
>>>
>>> I didn't change anything with jena cmd tools, to be honest, I never used
>> it
>>> before, so I just googled what to do and how to do it, and the tdbstats
>>> command was the one that made more sense. Is this a flag that is passed
>> in
>>> to tdbstats?
>>>
>>> Thanks,
>>> Antero
>>>
>>> On Tue, 5 Apr 2016 8:12 pm A. Soroka, <aj...@virginia.edu> wrote:
>>>
>>>> I don't know whether you will be able to restart the indexer, but it
>> seems
>>>> like you aren't getting much of anywhere so you might as well stop it
>> with
>>>> the proviso that you might have to start again from scratch. But YMMV.
>>>>
>>>> As for the Jena side of things: did you adjust the heap allocation when
>>>> you ran tdbstats?
>>>>
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>>
>>>>> On Apr 5, 2016, at 12:45 PM, Antero Duarte <a....@gmail.com>
>> wrote:
>>>>>
>>>>> Thanks for your reply.
>>>>>
>>>>> I downloaded jena to use the command line tools, what command should I
>>>> run?
>>>>> I tried running `tdbstats --loc=./tdb` (from the resources dir) but
>>>> after a
>>>>> whileit threw an exception because it ran out of heap space. Is there
>>>>> another command that can be more useful and doesn't use as much memory?
>>>> On
>>>>> the other hand if I can stop the indexing tool, I can clear some memory
>>>> to
>>>>> let tdbstats run.
>>>>>
>>>>> Regards,
>>>>> Antero
>>>>>
>>>>> On Tue, 5 Apr 2016 at 15:21 A. Soroka <aj...@virginia.edu> wrote:
>>>>>
>>>>>> Looks similar to something others have seen:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/STANBOL-1446
>>>>>>
>>>>>> which doesn't help you much, but might be a place to centralize the
>>>> answer
>>>>>> to this question. I wouldn't think that a WARN level message would
>> tag a
>>>>>> condition so severe that indexing doesn't take place. Perhaps it is
>>>>>> something else.
>>>>>>
>>>>>> Can you use Jena's command-line tools to check and see how many
>> entities
>>>>>> have actually been loaded into TDB vs. how many you expect? That might
>>>> give
>>>>>> you a clue as to where indexing is hanging up (if it actually is).
>>>>>>
>>>>>> ---
>>>>>> A. Soroka
>>>>>> The University of Virginia Library
>>>>>>
>>>>>>> On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>> Hello there,
>>>>>>>
>>>>>>> I have been struggling with building indexes from generic rdf and
>> even
>>>>>>> using default configuration for more popular sources like dbpedia.
>>>>>>>
>>>>>>> I found an indexing tool online configured to index yago, at
>>>>>>> https://github.com/ChalithaUdara/Stanbol-Yago-Site.
>>>>>>>
>>>>>>> Everything seemed to be going well until it got into this loop:
>>>>>>>
>>>>>>> 11:17:26,546 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
>>>>>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping
>> ignored!
>>>>>>> 11:17:26,546 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
>>>>>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
>>>>>>> 11:17:26,576 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
>>>>>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>>>>>>> 12:17:26,856 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
>>>>>>> http://prefix.cc/nsogi:' invalid -> mapping ignored!
>>>>>>> 12:17:26,918 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
>>>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>>>> 12:17:26,949 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'category' valid , namespace '
>>>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>>>> 12:17:26,949 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
>>>>>>> http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
>>>>>>> 12:17:26,950 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
>>>>>>> http://bio2rdf.org/chebi:' invalid -> mapping ignored!
>>>>>>> 12:17:26,980 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
>>>>>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
>>>>>>> 12:17:26,980 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
>>>>>>> http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
>>>>>>> 12:17:26,980 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
>>>>>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
>>>>>>> 12:17:26,981 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
>>>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>>>> 12:17:26,981 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'call' valid , namespace '
>>>>>>> http://webofcode.org/wfn/call:' invalid -> mapping ignored!
>>>>>>> 12:17:27,011 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
>>>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>>>> 12:17:27,011 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
>>>>>>> http://bg.dbpedia.org/resource/?????????:' invalid -> mapping
>> ignored!
>>>>>>> 12:17:27,012 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
>>>>>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping
>> ignored!
>>>>>>> 12:17:27,012 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
>>>>>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
>>>>>>> 12:17:27,042 [pool-1-thread-1] WARN
>> impl.NamespacePrefixProviderImpl -
>>>>>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
>>>>>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>>>>>>>
>>>>>>> It happened to me before with the dbpedia index and at first I
>> thought
>>>> it
>>>>>>> was some problem with the rdf source, and since theses messages are
>>>>>> logged
>>>>>>> at WARN level, I simply ignored them. but after days, the
>> indexing/tdb
>>>>>>> directory stayed the same size even though there are still files in
>> the
>>>>>>> indexing/resources/rdfdata directory. Then I realised that these
>>>> messages
>>>>>>> follow a pattern and they are logged every hour with precision to the
>>>>>>> second, which seems weird. Also, they are always the same messages.
>>>> This
>>>>>>> led me to think that the indexing tool is stuck in a loop and that's
>>>> why
>>>>>> it
>>>>>>> is not moving any further. I think it is important to say that the
>> one
>>>>>> hour
>>>>>>> time span between messages is the same for the dbpedia index and for
>>>> the
>>>>>>> yago index, the yago index is much bigger.
>>>>>>>
>>>>>>> I have been constantly running `watch du * -s` in the resources
>>>> directory
>>>>>>> for days to check for size changes and nothing is changing and hasn't
>>>>>>> changed for days.
>>>>>>>
>>>>>>> I don't know if this is some problem with the configuration, but
>> since
>>>> I
>>>>>>> didn't configure it myself, I assumed that what I got from github
>> would
>>>>>> be
>>>>>>> a working configuration for this specific index.
>>>>>>>
>>>>>>> I have a few questions related to this problem:
>>>>>>>
>>>>>>> 1) Is it safe to cancel the indexing tool and start again without
>>>>>> changing
>>>>>>> what's in the rdfdata and imported directories? Could this help at
>> all?
>>>>>>>
>>>>>>> 2) What can possibly be causing this problem?
>>>>>>>
>>>>>>> 3) Why is it looping and logging every hour (accurate to the second)?
>>>>>>>
>>>>>>> If there is any extra information I can provide that would help
>>>>>>> understanding what the problem is here, tell me what it is and I will
>>>>>>> provide it.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Antero
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: Indexing generic rdf: yago
Posted by Antero Duarte <a....@gmail.com>.
Hi,
Right, as an update to my last message, I changed the heap size for
tdbstats and redirected the output to a file which has 1,005,822 lines
(output of "wc -l"). Now if this directly relates to the number of
entities, I don't think it finished because the initial size of the files
in rdfdata/ was around 70GB.
One interesting thing though, the first time I ran tdbstats, the size of
the tdb directory increased... Not much and only once, it has been stable
ever since. Because I never used tdbstats before, I don't know if this is
the indexing tool showing it's still running or just tdbstats itself
changing something in the directory.
I restarted the indexing tool and the size of tdb seems to be increasing
again and the tool seems to be working well... Could it be that I just
needed to clear whatever was in RAM because it is a huge index?
I will see how it goes from now on and if further problems arise I will ask
again.
Thank you for your help.
Antero
On Wed, 6 Apr 2016 at 20:04 A. Soroka <aj...@virginia.edu> wrote:
> TDB command-line tools should respond to ordinary usage of the JAVA_OPTS
> environment variable, so you should be able to do something like:
>
> JAVA_OPTS="-Xmx2G" tdbstats --loc=./tdb
>
> to run with a 2GB heap, for example.
>
> One thing to note is that if you were even able to access the TDB database
> at all (which it seems you were because your attempt to use tdbstats didn't
> error out instantly) it means that no other process was using the database,
> particularly that the indexer was not. Only one system process can use a
> TDB database at a time. Now, that may or may not mean that the indexer was
> actually done doing stuff with the database: I don't know enough about how
> it works to know that, although the Stanbol devs on here would. It just
> means that at the moment you tried to run tdbstats, the indexer had
> relinquished control of the TDB database, at least for the moment.
>
> ---
> A. Soroka
> The University of Virginia Library
>
> > On Apr 5, 2016, at 6:42 PM, Antero Duarte <a....@gmail.com> wrote:
> >
> > Fair enough... I just wanted to figure out if it is actually stopped
> before
> > stopping it, but I will do that tomorrow.
> >
> > I didn't change anything with jena cmd tools, to be honest, I never used
> it
> > before, so I just googled what to do and how to do it, and the tdbstats
> > command was the one that made more sense. Is this a flag that is passed
> in
> > to tdbstats?
> >
> > Thanks,
> > Antero
> >
> > On Tue, 5 Apr 2016 8:12 pm A. Soroka, <aj...@virginia.edu> wrote:
> >
> >> I don't know whether you will be able to restart the indexer, but it
> seems
> >> like you aren't getting much of anywhere so you might as well stop it
> with
> >> the proviso that you might have to start again from scratch. But YMMV.
> >>
> >> As for the Jena side of things: did you adjust the heap allocation when
> >> you ran tdbstats?
> >>
> >> ---
> >> A. Soroka
> >> The University of Virginia Library
> >>
> >>> On Apr 5, 2016, at 12:45 PM, Antero Duarte <a....@gmail.com>
> wrote:
> >>>
> >>> Thanks for your reply.
> >>>
> >>> I downloaded jena to use the command line tools, what command should I
> >> run?
> >>> I tried running `tdbstats --loc=./tdb` (from the resources dir) but
> >> after a
> >>> whileit threw an exception because it ran out of heap space. Is there
> >>> another command that can be more useful and doesn't use as much memory?
> >> On
> >>> the other hand if I can stop the indexing tool, I can clear some memory
> >> to
> >>> let tdbstats run.
> >>>
> >>> Regards,
> >>> Antero
> >>>
> >>> On Tue, 5 Apr 2016 at 15:21 A. Soroka <aj...@virginia.edu> wrote:
> >>>
> >>>> Looks similar to something others have seen:
> >>>>
> >>>> https://issues.apache.org/jira/browse/STANBOL-1446
> >>>>
> >>>> which doesn't help you much, but might be a place to centralize the
> >> answer
> >>>> to this question. I wouldn't think that a WARN level message would
> tag a
> >>>> condition so severe that indexing doesn't take place. Perhaps it is
> >>>> something else.
> >>>>
> >>>> Can you use Jena's command-line tools to check and see how many
> entities
> >>>> have actually been loaded into TDB vs. how many you expect? That might
> >> give
> >>>> you a clue as to where indexing is hanging up (if it actually is).
> >>>>
> >>>> ---
> >>>> A. Soroka
> >>>> The University of Virginia Library
> >>>>
> >>>>> On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com>
> >> wrote:
> >>>>>
> >>>>> Hello there,
> >>>>>
> >>>>> I have been struggling with building indexes from generic rdf and
> even
> >>>>> using default configuration for more popular sources like dbpedia.
> >>>>>
> >>>>> I found an indexing tool online configured to index yago, at
> >>>>> https://github.com/ChalithaUdara/Stanbol-Yago-Site.
> >>>>>
> >>>>> Everything seemed to be going well until it got into this loop:
> >>>>>
> >>>>> 11:17:26,546 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> >>>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping
> ignored!
> >>>>> 11:17:26,546 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> >>>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
> >>>>> 11:17:26,576 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> >>>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> >>>>> 12:17:26,856 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
> >>>>> http://prefix.cc/nsogi:' invalid -> mapping ignored!
> >>>>> 12:17:26,918 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
> >>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>>>> 12:17:26,949 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'category' valid , namespace '
> >>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>>>> 12:17:26,949 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
> >>>>> http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
> >>>>> 12:17:26,950 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
> >>>>> http://bio2rdf.org/chebi:' invalid -> mapping ignored!
> >>>>> 12:17:26,980 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
> >>>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> >>>>> 12:17:26,980 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
> >>>>> http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
> >>>>> 12:17:26,980 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
> >>>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> >>>>> 12:17:26,981 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
> >>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>>>> 12:17:26,981 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'call' valid , namespace '
> >>>>> http://webofcode.org/wfn/call:' invalid -> mapping ignored!
> >>>>> 12:17:27,011 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
> >>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>>>> 12:17:27,011 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
> >>>>> http://bg.dbpedia.org/resource/?????????:' invalid -> mapping
> ignored!
> >>>>> 12:17:27,012 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> >>>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping
> ignored!
> >>>>> 12:17:27,012 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> >>>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
> >>>>> 12:17:27,042 [pool-1-thread-1] WARN
> impl.NamespacePrefixProviderImpl -
> >>>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> >>>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> >>>>>
> >>>>> It happened to me before with the dbpedia index and at first I
> thought
> >> it
> >>>>> was some problem with the rdf source, and since theses messages are
> >>>> logged
> >>>>> at WARN level, I simply ignored them. but after days, the
> indexing/tdb
> >>>>> directory stayed the same size even though there are still files in
> the
> >>>>> indexing/resources/rdfdata directory. Then I realised that these
> >> messages
> >>>>> follow a pattern and they are logged every hour with precision to the
> >>>>> second, which seems weird. Also, they are always the same messages.
> >> This
> >>>>> led me to think that the indexing tool is stuck in a loop and that's
> >> why
> >>>> it
> >>>>> is not moving any further. I think it is important to say that the
> one
> >>>> hour
> >>>>> time span between messages is the same for the dbpedia index and for
> >> the
> >>>>> yago index, the yago index is much bigger.
> >>>>>
> >>>>> I have been constantly running `watch du * -s` in the resources
> >> directory
> >>>>> for days to check for size changes and nothing is changing and hasn't
> >>>>> changed for days.
> >>>>>
> >>>>> I don't know if this is some problem with the configuration, but
> since
> >> I
> >>>>> didn't configure it myself, I assumed that what I got from github
> would
> >>>> be
> >>>>> a working configuration for this specific index.
> >>>>>
> >>>>> I have a few questions related to this problem:
> >>>>>
> >>>>> 1) Is it safe to cancel the indexing tool and start again without
> >>>> changing
> >>>>> what's in the rdfdata and imported directories? Could this help at
> all?
> >>>>>
> >>>>> 2) What can possibly be causing this problem?
> >>>>>
> >>>>> 3) Why is it looping and logging every hour (accurate to the second)?
> >>>>>
> >>>>> If there is any extra information I can provide that would help
> >>>>> understanding what the problem is here, tell me what it is and I will
> >>>>> provide it.
> >>>>>
> >>>>> Regards,
> >>>>> Antero
> >>>>
> >>>>
> >>
> >>
>
>
Re: Indexing generic rdf: yago
Posted by "A. Soroka" <aj...@virginia.edu>.
TDB command-line tools should respond to ordinary usage of the JAVA_OPTS environment variable, so you should be able to do something like:
JAVA_OPTS="-Xmx2G" tdbstats --loc=./tdb
to run with a 2GB heap, for example.
One thing to note is that if you were even able to access the TDB database at all (which it seems you were because your attempt to use tdbstats didn't error out instantly) it means that no other process was using the database, particularly that the indexer was not. Only one system process can use a TDB database at a time. Now, that may or may not mean that the indexer was actually done doing stuff with the database: I don't know enough about how it works to know that, although the Stanbol devs on here would. It just means that at the moment you tried to run tdbstats, the indexer had relinquished control of the TDB database, at least for the moment.
---
A. Soroka
The University of Virginia Library
> On Apr 5, 2016, at 6:42 PM, Antero Duarte <a....@gmail.com> wrote:
>
> Fair enough... I just wanted to figure out if it is actually stopped before
> stopping it, but I will do that tomorrow.
>
> I didn't change anything with jena cmd tools, to be honest, I never used it
> before, so I just googled what to do and how to do it, and the tdbstats
> command was the one that made more sense. Is this a flag that is passed in
> to tdbstats?
>
> Thanks,
> Antero
>
> On Tue, 5 Apr 2016 8:12 pm A. Soroka, <aj...@virginia.edu> wrote:
>
>> I don't know whether you will be able to restart the indexer, but it seems
>> like you aren't getting much of anywhere so you might as well stop it with
>> the proviso that you might have to start again from scratch. But YMMV.
>>
>> As for the Jena side of things: did you adjust the heap allocation when
>> you ran tdbstats?
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>> On Apr 5, 2016, at 12:45 PM, Antero Duarte <a....@gmail.com> wrote:
>>>
>>> Thanks for your reply.
>>>
>>> I downloaded jena to use the command line tools, what command should I
>> run?
>>> I tried running `tdbstats --loc=./tdb` (from the resources dir) but
>> after a
>>> whileit threw an exception because it ran out of heap space. Is there
>>> another command that can be more useful and doesn't use as much memory?
>> On
>>> the other hand if I can stop the indexing tool, I can clear some memory
>> to
>>> let tdbstats run.
>>>
>>> Regards,
>>> Antero
>>>
>>> On Tue, 5 Apr 2016 at 15:21 A. Soroka <aj...@virginia.edu> wrote:
>>>
>>>> Looks similar to something others have seen:
>>>>
>>>> https://issues.apache.org/jira/browse/STANBOL-1446
>>>>
>>>> which doesn't help you much, but might be a place to centralize the
>> answer
>>>> to this question. I wouldn't think that a WARN level message would tag a
>>>> condition so severe that indexing doesn't take place. Perhaps it is
>>>> something else.
>>>>
>>>> Can you use Jena's command-line tools to check and see how many entities
>>>> have actually been loaded into TDB vs. how many you expect? That might
>> give
>>>> you a clue as to where indexing is hanging up (if it actually is).
>>>>
>>>> ---
>>>> A. Soroka
>>>> The University of Virginia Library
>>>>
>>>>> On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com>
>> wrote:
>>>>>
>>>>> Hello there,
>>>>>
>>>>> I have been struggling with building indexes from generic rdf and even
>>>>> using default configuration for more popular sources like dbpedia.
>>>>>
>>>>> I found an indexing tool online configured to index yago, at
>>>>> https://github.com/ChalithaUdara/Stanbol-Yago-Site.
>>>>>
>>>>> Everything seemed to be going well until it got into this loop:
>>>>>
>>>>> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
>>>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
>>>>> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
>>>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
>>>>> 11:17:26,576 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
>>>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>>>>> 12:17:26,856 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
>>>>> http://prefix.cc/nsogi:' invalid -> mapping ignored!
>>>>> 12:17:26,918 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'category' valid , namespace '
>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
>>>>> http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
>>>>> 12:17:26,950 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
>>>>> http://bio2rdf.org/chebi:' invalid -> mapping ignored!
>>>>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
>>>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
>>>>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
>>>>> http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
>>>>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
>>>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
>>>>> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'call' valid , namespace '
>>>>> http://webofcode.org/wfn/call:' invalid -> mapping ignored!
>>>>> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
>>>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>>>> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
>>>>> http://bg.dbpedia.org/resource/?????????:' invalid -> mapping ignored!
>>>>> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
>>>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
>>>>> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
>>>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
>>>>> 12:17:27,042 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
>>>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>>>>>
>>>>> It happened to me before with the dbpedia index and at first I thought
>> it
>>>>> was some problem with the rdf source, and since theses messages are
>>>> logged
>>>>> at WARN level, I simply ignored them. but after days, the indexing/tdb
>>>>> directory stayed the same size even though there are still files in the
>>>>> indexing/resources/rdfdata directory. Then I realised that these
>> messages
>>>>> follow a pattern and they are logged every hour with precision to the
>>>>> second, which seems weird. Also, they are always the same messages.
>> This
>>>>> led me to think that the indexing tool is stuck in a loop and that's
>> why
>>>> it
>>>>> is not moving any further. I think it is important to say that the one
>>>> hour
>>>>> time span between messages is the same for the dbpedia index and for
>> the
>>>>> yago index, the yago index is much bigger.
>>>>>
>>>>> I have been constantly running `watch du * -s` in the resources
>> directory
>>>>> for days to check for size changes and nothing is changing and hasn't
>>>>> changed for days.
>>>>>
>>>>> I don't know if this is some problem with the configuration, but since
>> I
>>>>> didn't configure it myself, I assumed that what I got from github would
>>>> be
>>>>> a working configuration for this specific index.
>>>>>
>>>>> I have a few questions related to this problem:
>>>>>
>>>>> 1) Is it safe to cancel the indexing tool and start again without
>>>> changing
>>>>> what's in the rdfdata and imported directories? Could this help at all?
>>>>>
>>>>> 2) What can possibly be causing this problem?
>>>>>
>>>>> 3) Why is it looping and logging every hour (accurate to the second)?
>>>>>
>>>>> If there is any extra information I can provide that would help
>>>>> understanding what the problem is here, tell me what it is and I will
>>>>> provide it.
>>>>>
>>>>> Regards,
>>>>> Antero
>>>>
>>>>
>>
>>
Re: Indexing generic rdf: yago
Posted by Antero Duarte <a....@gmail.com>.
Fair enough... I just wanted to figure out if it is actually stopped before
stopping it, but I will do that tomorrow.
I didn't change anything with jena cmd tools, to be honest, I never used it
before, so I just googled what to do and how to do it, and the tdbstats
command was the one that made more sense. Is this a flag that is passed in
to tdbstats?
Thanks,
Antero
On Tue, 5 Apr 2016 8:12 pm A. Soroka, <aj...@virginia.edu> wrote:
> I don't know whether you will be able to restart the indexer, but it seems
> like you aren't getting much of anywhere so you might as well stop it with
> the proviso that you might have to start again from scratch. But YMMV.
>
> As for the Jena side of things: did you adjust the heap allocation when
> you ran tdbstats?
>
> ---
> A. Soroka
> The University of Virginia Library
>
> > On Apr 5, 2016, at 12:45 PM, Antero Duarte <a....@gmail.com> wrote:
> >
> > Thanks for your reply.
> >
> > I downloaded jena to use the command line tools, what command should I
> run?
> > I tried running `tdbstats --loc=./tdb` (from the resources dir) but
> after a
> > whileit threw an exception because it ran out of heap space. Is there
> > another command that can be more useful and doesn't use as much memory?
> On
> > the other hand if I can stop the indexing tool, I can clear some memory
> to
> > let tdbstats run.
> >
> > Regards,
> > Antero
> >
> > On Tue, 5 Apr 2016 at 15:21 A. Soroka <aj...@virginia.edu> wrote:
> >
> >> Looks similar to something others have seen:
> >>
> >> https://issues.apache.org/jira/browse/STANBOL-1446
> >>
> >> which doesn't help you much, but might be a place to centralize the
> answer
> >> to this question. I wouldn't think that a WARN level message would tag a
> >> condition so severe that indexing doesn't take place. Perhaps it is
> >> something else.
> >>
> >> Can you use Jena's command-line tools to check and see how many entities
> >> have actually been loaded into TDB vs. how many you expect? That might
> give
> >> you a clue as to where indexing is hanging up (if it actually is).
> >>
> >> ---
> >> A. Soroka
> >> The University of Virginia Library
> >>
> >>> On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com>
> wrote:
> >>>
> >>> Hello there,
> >>>
> >>> I have been struggling with building indexes from generic rdf and even
> >>> using default configuration for more popular sources like dbpedia.
> >>>
> >>> I found an indexing tool online configured to index yago, at
> >>> https://github.com/ChalithaUdara/Stanbol-Yago-Site.
> >>>
> >>> Everything seemed to be going well until it got into this loop:
> >>>
> >>> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> >>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
> >>> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> >>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
> >>> 11:17:26,576 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> >>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> >>> 12:17:26,856 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
> >>> http://prefix.cc/nsogi:' invalid -> mapping ignored!
> >>> 12:17:26,918 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
> >>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'category' valid , namespace '
> >>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
> >>> http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
> >>> 12:17:26,950 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
> >>> http://bio2rdf.org/chebi:' invalid -> mapping ignored!
> >>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
> >>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> >>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
> >>> http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
> >>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
> >>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> >>> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
> >>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'call' valid , namespace '
> >>> http://webofcode.org/wfn/call:' invalid -> mapping ignored!
> >>> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
> >>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> >>> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
> >>> http://bg.dbpedia.org/resource/?????????:' invalid -> mapping ignored!
> >>> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> >>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
> >>> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> >>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
> >>> 12:17:27,042 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> >>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> >>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> >>>
> >>> It happened to me before with the dbpedia index and at first I thought
> it
> >>> was some problem with the rdf source, and since theses messages are
> >> logged
> >>> at WARN level, I simply ignored them. but after days, the indexing/tdb
> >>> directory stayed the same size even though there are still files in the
> >>> indexing/resources/rdfdata directory. Then I realised that these
> messages
> >>> follow a pattern and they are logged every hour with precision to the
> >>> second, which seems weird. Also, they are always the same messages.
> This
> >>> led me to think that the indexing tool is stuck in a loop and that's
> why
> >> it
> >>> is not moving any further. I think it is important to say that the one
> >> hour
> >>> time span between messages is the same for the dbpedia index and for
> the
> >>> yago index, the yago index is much bigger.
> >>>
> >>> I have been constantly running `watch du * -s` in the resources
> directory
> >>> for days to check for size changes and nothing is changing and hasn't
> >>> changed for days.
> >>>
> >>> I don't know if this is some problem with the configuration, but since
> I
> >>> didn't configure it myself, I assumed that what I got from github would
> >> be
> >>> a working configuration for this specific index.
> >>>
> >>> I have a few questions related to this problem:
> >>>
> >>> 1) Is it safe to cancel the indexing tool and start again without
> >> changing
> >>> what's in the rdfdata and imported directories? Could this help at all?
> >>>
> >>> 2) What can possibly be causing this problem?
> >>>
> >>> 3) Why is it looping and logging every hour (accurate to the second)?
> >>>
> >>> If there is any extra information I can provide that would help
> >>> understanding what the problem is here, tell me what it is and I will
> >>> provide it.
> >>>
> >>> Regards,
> >>> Antero
> >>
> >>
>
>
Re: Indexing generic rdf: yago
Posted by "A. Soroka" <aj...@virginia.edu>.
I don't know whether you will be able to restart the indexer, but it seems like you aren't getting much of anywhere so you might as well stop it with the proviso that you might have to start again from scratch. But YMMV.
As for the Jena side of things: did you adjust the heap allocation when you ran tdbstats?
---
A. Soroka
The University of Virginia Library
> On Apr 5, 2016, at 12:45 PM, Antero Duarte <a....@gmail.com> wrote:
>
> Thanks for your reply.
>
> I downloaded jena to use the command line tools, what command should I run?
> I tried running `tdbstats --loc=./tdb` (from the resources dir) but after a
> whileit threw an exception because it ran out of heap space. Is there
> another command that can be more useful and doesn't use as much memory? On
> the other hand if I can stop the indexing tool, I can clear some memory to
> let tdbstats run.
>
> Regards,
> Antero
>
> On Tue, 5 Apr 2016 at 15:21 A. Soroka <aj...@virginia.edu> wrote:
>
>> Looks similar to something others have seen:
>>
>> https://issues.apache.org/jira/browse/STANBOL-1446
>>
>> which doesn't help you much, but might be a place to centralize the answer
>> to this question. I wouldn't think that a WARN level message would tag a
>> condition so severe that indexing doesn't take place. Perhaps it is
>> something else.
>>
>> Can you use Jena's command-line tools to check and see how many entities
>> have actually been loaded into TDB vs. how many you expect? That might give
>> you a clue as to where indexing is hanging up (if it actually is).
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>> On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com> wrote:
>>>
>>> Hello there,
>>>
>>> I have been struggling with building indexes from generic rdf and even
>>> using default configuration for more popular sources like dbpedia.
>>>
>>> I found an indexing tool online configured to index yago, at
>>> https://github.com/ChalithaUdara/Stanbol-Yago-Site.
>>>
>>> Everything seemed to be going well until it got into this loop:
>>>
>>> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
>>> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
>>> 11:17:26,576 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>>> 12:17:26,856 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
>>> http://prefix.cc/nsogi:' invalid -> mapping ignored!
>>> 12:17:26,918 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'category' valid , namespace '
>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
>>> http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
>>> 12:17:26,950 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
>>> http://bio2rdf.org/chebi:' invalid -> mapping ignored!
>>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
>>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
>>> http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
>>> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
>>> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
>>> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'call' valid , namespace '
>>> http://webofcode.org/wfn/call:' invalid -> mapping ignored!
>>> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
>>> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
>>> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
>>> http://bg.dbpedia.org/resource/?????????:' invalid -> mapping ignored!
>>> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
>>> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
>>> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
>>> http://www.kinjal.com/condition:' invalid -> mapping ignored!
>>> 12:17:27,042 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
>>> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
>>> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>>>
>>> It happened to me before with the dbpedia index and at first I thought it
>>> was some problem with the rdf source, and since theses messages are
>> logged
>>> at WARN level, I simply ignored them. but after days, the indexing/tdb
>>> directory stayed the same size even though there are still files in the
>>> indexing/resources/rdfdata directory. Then I realised that these messages
>>> follow a pattern and they are logged every hour with precision to the
>>> second, which seems weird. Also, they are always the same messages. This
>>> led me to think that the indexing tool is stuck in a loop and that's why
>> it
>>> is not moving any further. I think it is important to say that the one
>> hour
>>> time span between messages is the same for the dbpedia index and for the
>>> yago index, the yago index is much bigger.
>>>
>>> I have been constantly running `watch du * -s` in the resources directory
>>> for days to check for size changes and nothing is changing and hasn't
>>> changed for days.
>>>
>>> I don't know if this is some problem with the configuration, but since I
>>> didn't configure it myself, I assumed that what I got from github would
>> be
>>> a working configuration for this specific index.
>>>
>>> I have a few questions related to this problem:
>>>
>>> 1) Is it safe to cancel the indexing tool and start again without
>> changing
>>> what's in the rdfdata and imported directories? Could this help at all?
>>>
>>> 2) What can possibly be causing this problem?
>>>
>>> 3) Why is it looping and logging every hour (accurate to the second)?
>>>
>>> If there is any extra information I can provide that would help
>>> understanding what the problem is here, tell me what it is and I will
>>> provide it.
>>>
>>> Regards,
>>> Antero
>>
>>
Re: Indexing generic rdf: yago
Posted by Antero Duarte <a....@gmail.com>.
Thanks for your reply.
I downloaded jena to use the command line tools, what command should I run?
I tried running `tdbstats --loc=./tdb` (from the resources dir) but after a
whileit threw an exception because it ran out of heap space. Is there
another command that can be more useful and doesn't use as much memory? On
the other hand if I can stop the indexing tool, I can clear some memory to
let tdbstats run.
Regards,
Antero
On Tue, 5 Apr 2016 at 15:21 A. Soroka <aj...@virginia.edu> wrote:
> Looks similar to something others have seen:
>
> https://issues.apache.org/jira/browse/STANBOL-1446
>
> which doesn't help you much, but might be a place to centralize the answer
> to this question. I wouldn't think that a WARN level message would tag a
> condition so severe that indexing doesn't take place. Perhaps it is
> something else.
>
> Can you use Jena's command-line tools to check and see how many entities
> have actually been loaded into TDB vs. how many you expect? That might give
> you a clue as to where indexing is hanging up (if it actually is).
>
> ---
> A. Soroka
> The University of Virginia Library
>
> > On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com> wrote:
> >
> > Hello there,
> >
> > I have been struggling with building indexes from generic rdf and even
> > using default configuration for more popular sources like dbpedia.
> >
> > I found an indexing tool online configured to index yago, at
> > https://github.com/ChalithaUdara/Stanbol-Yago-Site.
> >
> > Everything seemed to be going well until it got into this loop:
> >
> > 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> > http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
> > 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> > http://www.kinjal.com/condition:' invalid -> mapping ignored!
> > 11:17:26,576 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> > http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> > 12:17:26,856 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
> > http://prefix.cc/nsogi:' invalid -> mapping ignored!
> > 12:17:26,918 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
> > http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> > 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'category' valid , namespace '
> > http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> > 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
> > http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
> > 12:17:26,950 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
> > http://bio2rdf.org/chebi:' invalid -> mapping ignored!
> > 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
> > http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> > 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
> > http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
> > 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
> > http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> > 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
> > http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> > 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'call' valid , namespace '
> > http://webofcode.org/wfn/call:' invalid -> mapping ignored!
> > 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
> > http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> > 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
> > http://bg.dbpedia.org/resource/?????????:' invalid -> mapping ignored!
> > 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> > http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
> > 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> > http://www.kinjal.com/condition:' invalid -> mapping ignored!
> > 12:17:27,042 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> > Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> > http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> >
> > It happened to me before with the dbpedia index and at first I thought it
> > was some problem with the rdf source, and since theses messages are
> logged
> > at WARN level, I simply ignored them. but after days, the indexing/tdb
> > directory stayed the same size even though there are still files in the
> > indexing/resources/rdfdata directory. Then I realised that these messages
> > follow a pattern and they are logged every hour with precision to the
> > second, which seems weird. Also, they are always the same messages. This
> > led me to think that the indexing tool is stuck in a loop and that's why
> it
> > is not moving any further. I think it is important to say that the one
> hour
> > time span between messages is the same for the dbpedia index and for the
> > yago index, the yago index is much bigger.
> >
> > I have been constantly running `watch du * -s` in the resources directory
> > for days to check for size changes and nothing is changing and hasn't
> > changed for days.
> >
> > I don't know if this is some problem with the configuration, but since I
> > didn't configure it myself, I assumed that what I got from github would
> be
> > a working configuration for this specific index.
> >
> > I have a few questions related to this problem:
> >
> > 1) Is it safe to cancel the indexing tool and start again without
> changing
> > what's in the rdfdata and imported directories? Could this help at all?
> >
> > 2) What can possibly be causing this problem?
> >
> > 3) Why is it looping and logging every hour (accurate to the second)?
> >
> > If there is any extra information I can provide that would help
> > understanding what the problem is here, tell me what it is and I will
> > provide it.
> >
> > Regards,
> > Antero
>
>
Re: Indexing generic rdf: yago
Posted by "A. Soroka" <aj...@virginia.edu>.
Looks similar to something others have seen:
https://issues.apache.org/jira/browse/STANBOL-1446
which doesn't help you much, but might be a place to centralize the answer to this question. I wouldn't think that a WARN level message would tag a condition so severe that indexing doesn't take place. Perhaps it is something else.
Can you use Jena's command-line tools to check and see how many entities have actually been loaded into TDB vs. how many you expect? That might give you a clue as to where indexing is hanging up (if it actually is).
---
A. Soroka
The University of Virginia Library
> On Apr 5, 2016, at 7:59 AM, Antero Duarte <a....@gmail.com> wrote:
>
> Hello there,
>
> I have been struggling with building indexes from generic rdf and even
> using default configuration for more popular sources like dbpedia.
>
> I found an indexing tool online configured to index yago, at
> https://github.com/ChalithaUdara/Stanbol-Yago-Site.
>
> Everything seemed to be going well until it got into this loop:
>
> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
> 11:17:26,546 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> http://www.kinjal.com/condition:' invalid -> mapping ignored!
> 11:17:26,576 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
> 12:17:26,856 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'nsogi' valid , namespace '
> http://prefix.cc/nsogi:' invalid -> mapping ignored!
> 12:17:26,918 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'dbc' valid , namespace '
> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'category' valid , namespace '
> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> 12:17:26,949 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'hgnc' valid , namespace '
> http://bio2rdf.org/hgnc:' invalid -> mapping ignored!
> 12:17:26,950 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'chebi' valid , namespace '
> http://bio2rdf.org/chebi:' invalid -> mapping ignored!
> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'dbt' valid , namespace '
> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'pubmed' valid , namespace '
> http://bio2rdf.org/pubmed_vocabulary:' invalid -> mapping ignored!
> 12:17:26,980 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'dbptmpl' valid , namespace '
> http://dbpedia.org/resource/Template:' invalid -> mapping ignored!
> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'dbrc' valid , namespace '
> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> 12:17:26,981 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'call' valid , namespace '
> http://webofcode.org/wfn/call:' invalid -> mapping ignored!
> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'dbcat' valid , namespace '
> http://dbpedia.org/resource/Category:' invalid -> mapping ignored!
> 12:17:27,011 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'bgcat' valid , namespace '
> http://bg.dbpedia.org/resource/?????????:' invalid -> mapping ignored!
> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'affymetrix' valid , namespace '
> http://bio2rdf.org/affymetrix_vocabulary:' invalid -> mapping ignored!
> 12:17:27,012 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'condition' valid , namespace '
> http://www.kinjal.com/condition:' invalid -> mapping ignored!
> 12:17:27,042 [pool-1-thread-1] WARN impl.NamespacePrefixProviderImpl -
> Invalid Namespace Mapping: prefix 'wimpo' valid , namespace '
> http://rdfex.org/withImports?uri=' invalid -> mapping ignored!
>
> It happened to me before with the dbpedia index and at first I thought it
> was some problem with the rdf source, and since theses messages are logged
> at WARN level, I simply ignored them. but after days, the indexing/tdb
> directory stayed the same size even though there are still files in the
> indexing/resources/rdfdata directory. Then I realised that these messages
> follow a pattern and they are logged every hour with precision to the
> second, which seems weird. Also, they are always the same messages. This
> led me to think that the indexing tool is stuck in a loop and that's why it
> is not moving any further. I think it is important to say that the one hour
> time span between messages is the same for the dbpedia index and for the
> yago index, the yago index is much bigger.
>
> I have been constantly running `watch du * -s` in the resources directory
> for days to check for size changes and nothing is changing and hasn't
> changed for days.
>
> I don't know if this is some problem with the configuration, but since I
> didn't configure it myself, I assumed that what I got from github would be
> a working configuration for this specific index.
>
> I have a few questions related to this problem:
>
> 1) Is it safe to cancel the indexing tool and start again without changing
> what's in the rdfdata and imported directories? Could this help at all?
>
> 2) What can possibly be causing this problem?
>
> 3) Why is it looping and logging every hour (accurate to the second)?
>
> If there is any extra information I can provide that would help
> understanding what the problem is here, tell me what it is and I will
> provide it.
>
> Regards,
> Antero