You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Mikael Pesonen <mi...@lingsoft.fi> on 2016/08/08 09:25:03 UTC
Slow SPARQL query
Hi,
I'm not if this is the correct forum to ask but hope you can help. This
query takes over 20 seconds with jena:
SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
SELECT DISTINCT ?child WHERE {
{<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION
{<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } }
GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
<http://purl.org/dc/terms/subject> ?child . ?s
<http://purl.org/dc/terms/isPartOf>
<http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph
query is for getting keywords from an ontology graph, second is for
querying documents having those keywords. Is there better way/order to
make this query? Thank you for the help, Mikael
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Any way thank you for help Håvard!
I might have to rethink the system so that we can use partial queries.
It's at least a sure way to make things quicker.
-Mikael
On 8.8.2016 15:33, Håvard Ottestad wrote:
> Hi Mikael
>
> I don't think I can be of much more help now. Maybe if you post the data somewhere then I could try the queries myself to see if I could make it faster.
>
> Håvard
>
>> On 08 Aug 2016, at 14:16, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>
>>
>> For some reason with filter I dont get any results (corrected == to = ). Without filter it takes 12 secs so about the same speed.
>>
>> -Mikael
>>
>>
>>> On 8.8.2016 14:40, Håvard Ottestad wrote:
>>> Maybe your inner select gets executed for every results of the second graph query. How about executing them separately and joining them with a filter?
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>> Union{
>>> GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } }
>>> }
>>> union {
>>> GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child2 . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o }
>>> }
>>> filter(?child == ?child2)
>>> }
>>>
>>> Håvard
>>>
>>>> On 08 Aug 2016, at 13:03, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>
>>>>
>>>> Also tested original query but removed all * from narrower, so ontology query returns less keywords, and query time dropped to half. Still too slow...
>>>>
>>>> -Mikael
>>>>
>>>>
>>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>> There is a a certain amount of "it depend" here: what's the data stored in? what shape is the data?( which Jena version?)
>>>>>
>>>>> In the next release, and available in development builds is:
>>>>>
>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>
>>>>> where property* got speeded up recently. Usually, it took moderately unusual data to show this up but the repeated use of an expensive operation in property* may be happening here too.
>>>>>
>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>>>>> Is this any better?
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>> ?s ?p ?o
>>>>>> }
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Regards,
>>>>>> Håvard M. Ottestad
>>>>>>
>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>>>>>>
>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>>>>>>
>>>>>>> --
>>>>>>> www.lingsoft.fi
>>>>>>>
>>>>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>
>>>>>>> Mikael Pesonen
>>>>>>> System Engineer
>>>>>>>
>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>> Tel. +358 2 279 3300
>>>>>>>
>>>>>>> Time zone: GMT+2
>>>>>>>
>>>>>>> Helsinki Office
>>>>>>> Eteläranta 10
>>>>>>> FI-00130 Helsinki
>>>>>>> FINLAND
>>>>>>>
>>>>>>> Turku Office
>>>>>>> Linnankatu 10 A
>>>>>>> FI-20100 Turku
>>>>>>> FINLAND
>>>> --
>>>> www.lingsoft.fi
>>>>
>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>
>>>> Mikael Pesonen
>>>> System Engineer
>>>>
>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>> Tel. +358 2 279 3300
>>>>
>>>> Time zone: GMT+2
>>>>
>>>> Helsinki Office
>>>> Eteläranta 10
>>>> FI-00130 Helsinki
>>>> FINLAND
>>>>
>>>> Turku Office
>>>> Linnankatu 10 A
>>>> FI-20100 Turku
>>>> FINLAND
>> --
>> www.lingsoft.fi
>>
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>
>> Mikael Pesonen
>> System Engineer
>>
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>>
>> Time zone: GMT+2
>>
>> Helsinki Office
>> Eteläranta 10
>> FI-00130 Helsinki
>> FINLAND
>>
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Håvard Ottestad <hm...@gmail.com>.
Hi Mikael
I don't think I can be of much more help now. Maybe if you post the data somewhere then I could try the queries myself to see if I could make it faster.
Håvard
> On 08 Aug 2016, at 14:16, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>
>
> For some reason with filter I dont get any results (corrected == to = ). Without filter it takes 12 secs so about the same speed.
>
> -Mikael
>
>
>> On 8.8.2016 14:40, Håvard Ottestad wrote:
>> Maybe your inner select gets executed for every results of the second graph query. How about executing them separately and joining them with a filter?
>>
>> SELECT DISTINCT ?s ?p ?o WHERE {
>> Union{
>> GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } }
>> }
>> union {
>> GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child2 . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o }
>> }
>> filter(?child == ?child2)
>> }
>>
>> Håvard
>>
>>> On 08 Aug 2016, at 13:03, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>
>>>
>>> Also tested original query but removed all * from narrower, so ontology query returns less keywords, and query time dropped to half. Still too slow...
>>>
>>> -Mikael
>>>
>>>
>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>> There is a a certain amount of "it depend" here: what's the data stored in? what shape is the data?( which Jena version?)
>>>>
>>>> In the next release, and available in development builds is:
>>>>
>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>
>>>> where property* got speeded up recently. Usually, it took moderately unusual data to show this up but the repeated use of an expensive operation in property* may be happening here too.
>>>>
>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>
>>>> Andy
>>>>
>>>>
>>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>>>> Is this any better?
>>>>>
>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>
>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>> ?s ?p ?o
>>>>> }
>>>>>
>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>> SELECT DISTINCT ?child WHERE {
>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>>>> }
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> Regards,
>>>>> Håvard M. Ottestad
>>>>>
>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>>>>>
>>>>>> --
>>>>>> www.lingsoft.fi
>>>>>>
>>>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>>>
>>>>>> Mikael Pesonen
>>>>>> System Engineer
>>>>>>
>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>> Tel. +358 2 279 3300
>>>>>>
>>>>>> Time zone: GMT+2
>>>>>>
>>>>>> Helsinki Office
>>>>>> Eteläranta 10
>>>>>> FI-00130 Helsinki
>>>>>> FINLAND
>>>>>>
>>>>>> Turku Office
>>>>>> Linnankatu 10 A
>>>>>> FI-20100 Turku
>>>>>> FINLAND
>>> --
>>> www.lingsoft.fi
>>>
>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>
>>> Mikael Pesonen
>>> System Engineer
>>>
>>> e-mail: mikael.pesonen@lingsoft.fi
>>> Tel. +358 2 279 3300
>>>
>>> Time zone: GMT+2
>>>
>>> Helsinki Office
>>> Eteläranta 10
>>> FI-00130 Helsinki
>>> FINLAND
>>>
>>> Turku Office
>>> Linnankatu 10 A
>>> FI-20100 Turku
>>> FINLAND
>
> --
> www.lingsoft.fi
>
> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>
> Mikael Pesonen
> System Engineer
>
> e-mail: mikael.pesonen@lingsoft.fi
> Tel. +358 2 279 3300
>
> Time zone: GMT+2
>
> Helsinki Office
> Eteläranta 10
> FI-00130 Helsinki
> FINLAND
>
> Turku Office
> Linnankatu 10 A
> FI-20100 Turku
> FINLAND
>
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
For some reason with filter I dont get any results (corrected == to = ).
Without filter it takes 12 secs so about the same speed.
-Mikael
On 8.8.2016 14:40, Håvard Ottestad wrote:
> Maybe your inner select gets executed for every results of the second graph query. How about executing them separately and joining them with a filter?
>
> SELECT DISTINCT ?s ?p ?o WHERE {
> Union{
> GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } }
> }
> union {
> GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child2 . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o }
> }
>
> filter(?child == ?child2)
>
> }
>
> Håvard
>
>> On 08 Aug 2016, at 13:03, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>
>>
>> Also tested original query but removed all * from narrower, so ontology query returns less keywords, and query time dropped to half. Still too slow...
>>
>> -Mikael
>>
>>
>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>> There is a a certain amount of "it depend" here: what's the data stored in? what shape is the data?( which Jena version?)
>>>
>>> In the next release, and available in development builds is:
>>>
>>> https://issues.apache.org/jira/browse/JENA-1195
>>>
>>> where property* got speeded up recently. Usually, it took moderately unusual data to show this up but the repeated use of an expensive operation in property* may be happening here too.
>>>
>>> Mikael - are you able to try out a SNAPSHOT build?
>>>
>>> Andy
>>>
>>>
>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>>> Is this any better?
>>>>
>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>
>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>> ?s ?p ?o
>>>> }
>>>>
>>>> GRAPH <http://www.lingsoft.fi/> {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>>> }
>>>> }
>>>>
>>>> }
>>>>
>>>> Regards,
>>>> Håvard M. Ottestad
>>>>
>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>>>>
>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>>>>
>>>>> --
>>>>> www.lingsoft.fi
>>>>>
>>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>>
>>>>> Mikael Pesonen
>>>>> System Engineer
>>>>>
>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>> Tel. +358 2 279 3300
>>>>>
>>>>> Time zone: GMT+2
>>>>>
>>>>> Helsinki Office
>>>>> Eteläranta 10
>>>>> FI-00130 Helsinki
>>>>> FINLAND
>>>>>
>>>>> Turku Office
>>>>> Linnankatu 10 A
>>>>> FI-20100 Turku
>>>>> FINLAND
>> --
>> www.lingsoft.fi
>>
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>
>> Mikael Pesonen
>> System Engineer
>>
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>>
>> Time zone: GMT+2
>>
>> Helsinki Office
>> Eteläranta 10
>> FI-00130 Helsinki
>> FINLAND
>>
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Håvard Ottestad <hm...@gmail.com>.
Maybe your inner select gets executed for every results of the second graph query. How about executing them separately and joining them with a filter?
SELECT DISTINCT ?s ?p ?o WHERE {
Union{
GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } }
}
union {
GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child2 . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o }
}
filter(?child == ?child2)
}
Håvard
> On 08 Aug 2016, at 13:03, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>
>
> Also tested original query but removed all * from narrower, so ontology query returns less keywords, and query time dropped to half. Still too slow...
>
> -Mikael
>
>
>> On 8.8.2016 13:43, Andy Seaborne wrote:
>> There is a a certain amount of "it depend" here: what's the data stored in? what shape is the data?( which Jena version?)
>>
>> In the next release, and available in development builds is:
>>
>> https://issues.apache.org/jira/browse/JENA-1195
>>
>> where property* got speeded up recently. Usually, it took moderately unusual data to show this up but the repeated use of an expensive operation in property* may be happening here too.
>>
>> Mikael - are you able to try out a SNAPSHOT build?
>>
>> Andy
>>
>>
>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>> Is this any better?
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>
>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>> ?s ?p ?o
>>> }
>>>
>>> GRAPH <http://www.lingsoft.fi/> {
>>> SELECT DISTINCT ?child WHERE {
>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>> }
>>> }
>>>
>>> }
>>>
>>> Regards,
>>> Håvard M. Ottestad
>>>
>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>>>
>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>>>
>>>> --
>>>> www.lingsoft.fi
>>>>
>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>
>>>> Mikael Pesonen
>>>> System Engineer
>>>>
>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>> Tel. +358 2 279 3300
>>>>
>>>> Time zone: GMT+2
>>>>
>>>> Helsinki Office
>>>> Eteläranta 10
>>>> FI-00130 Helsinki
>>>> FINLAND
>>>>
>>>> Turku Office
>>>> Linnankatu 10 A
>>>> FI-20100 Turku
>>>> FINLAND
>
> --
> www.lingsoft.fi
>
> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>
> Mikael Pesonen
> System Engineer
>
> e-mail: mikael.pesonen@lingsoft.fi
> Tel. +358 2 279 3300
>
> Time zone: GMT+2
>
> Helsinki Office
> Eteläranta 10
> FI-00130 Helsinki
> FINLAND
>
> Turku Office
> Linnankatu 10 A
> FI-20100 Turku
> FINLAND
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
It takes 8 seconds, so about half of the time. It returns ~16k results.
I'm guessing getting this faster is very hard on software level?
Entire graph contains ~400k triplets.
-Mikael
On 8.8.2016 14:30, Håvard Ottestad wrote:
> How long does this part take? And how many results are you getting?
>
> SELECT DISTINCT ?s ?p ?o WHERE {
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
> ?s <http://purl.org/dc/terms/subject> ?child .
> ?s ?p ?o
> }
> }
>
> Håvard
>
>> On 08 Aug 2016, at 13:03, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>
>>
>> Also tested original query but removed all * from narrower, so ontology query returns less keywords, and query time dropped to half. Still too slow...
>>
>> -Mikael
>>
>>
>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>> There is a a certain amount of "it depend" here: what's the data stored in? what shape is the data?( which Jena version?)
>>>
>>> In the next release, and available in development builds is:
>>>
>>> https://issues.apache.org/jira/browse/JENA-1195
>>>
>>> where property* got speeded up recently. Usually, it took moderately unusual data to show this up but the repeated use of an expensive operation in property* may be happening here too.
>>>
>>> Mikael - are you able to try out a SNAPSHOT build?
>>>
>>> Andy
>>>
>>>
>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>>> Is this any better?
>>>>
>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>
>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>> ?s ?p ?o
>>>> }
>>>>
>>>> GRAPH <http://www.lingsoft.fi/> {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>>> }
>>>> }
>>>>
>>>> }
>>>>
>>>> Regards,
>>>> Håvard M. Ottestad
>>>>
>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>>>>
>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>>>>
>>>>> --
>>>>> www.lingsoft.fi
>>>>>
>>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>>
>>>>> Mikael Pesonen
>>>>> System Engineer
>>>>>
>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>> Tel. +358 2 279 3300
>>>>>
>>>>> Time zone: GMT+2
>>>>>
>>>>> Helsinki Office
>>>>> Eteläranta 10
>>>>> FI-00130 Helsinki
>>>>> FINLAND
>>>>>
>>>>> Turku Office
>>>>> Linnankatu 10 A
>>>>> FI-20100 Turku
>>>>> FINLAND
>> --
>> www.lingsoft.fi
>>
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>
>> Mikael Pesonen
>> System Engineer
>>
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>>
>> Time zone: GMT+2
>>
>> Helsinki Office
>> Eteläranta 10
>> FI-00130 Helsinki
>> FINLAND
>>
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Håvard Ottestad <hm...@gmail.com>.
How long does this part take? And how many results are you getting?
SELECT DISTINCT ?s ?p ?o WHERE {
GRAPH <http://www.lingsoft.fi/resource-meta/> {
?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
?s <http://purl.org/dc/terms/subject> ?child .
?s ?p ?o
}
}
Håvard
> On 08 Aug 2016, at 13:03, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>
>
> Also tested original query but removed all * from narrower, so ontology query returns less keywords, and query time dropped to half. Still too slow...
>
> -Mikael
>
>
>> On 8.8.2016 13:43, Andy Seaborne wrote:
>> There is a a certain amount of "it depend" here: what's the data stored in? what shape is the data?( which Jena version?)
>>
>> In the next release, and available in development builds is:
>>
>> https://issues.apache.org/jira/browse/JENA-1195
>>
>> where property* got speeded up recently. Usually, it took moderately unusual data to show this up but the repeated use of an expensive operation in property* may be happening here too.
>>
>> Mikael - are you able to try out a SNAPSHOT build?
>>
>> Andy
>>
>>
>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>> Is this any better?
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>
>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>> ?s ?p ?o
>>> }
>>>
>>> GRAPH <http://www.lingsoft.fi/> {
>>> SELECT DISTINCT ?child WHERE {
>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>> }
>>> }
>>>
>>> }
>>>
>>> Regards,
>>> Håvard M. Ottestad
>>>
>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>>>
>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>>>
>>>> --
>>>> www.lingsoft.fi
>>>>
>>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>>>
>>>> Mikael Pesonen
>>>> System Engineer
>>>>
>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>> Tel. +358 2 279 3300
>>>>
>>>> Time zone: GMT+2
>>>>
>>>> Helsinki Office
>>>> Eteläranta 10
>>>> FI-00130 Helsinki
>>>> FINLAND
>>>>
>>>> Turku Office
>>>> Linnankatu 10 A
>>>> FI-20100 Turku
>>>> FINLAND
>
> --
> www.lingsoft.fi
>
> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>
> Mikael Pesonen
> System Engineer
>
> e-mail: mikael.pesonen@lingsoft.fi
> Tel. +358 2 279 3300
>
> Time zone: GMT+2
>
> Helsinki Office
> Eteläranta 10
> FI-00130 Helsinki
> FINLAND
>
> Turku Office
> Linnankatu 10 A
> FI-20100 Turku
> FINLAND
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Also tested original query but removed all * from narrower, so ontology
query returns less keywords, and query time dropped to half. Still too
slow...
-Mikael
On 8.8.2016 13:43, Andy Seaborne wrote:
> There is a a certain amount of "it depend" here: what's the data
> stored in? what shape is the data?( which Jena version?)
>
> In the next release, and available in development builds is:
>
> https://issues.apache.org/jira/browse/JENA-1195
>
> where property* got speeded up recently. Usually, it took moderately
> unusual data to show this up but the repeated use of an expensive
> operation in property* may be happening here too.
>
> Mikael - are you able to try out a SNAPSHOT build?
>
> Andy
>
>
> On 08/08/16 11:37, H�vard Ottestad wrote:
>> Is this any better?
>>
>> SELECT DISTINCT ?s ?p ?o WHERE {
>>
>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>> ?s <http://purl.org/dc/terms/isPartOf>
>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>> ?s <http://purl.org/dc/terms/subject> ?child .
>> ?s ?p ?o
>> }
>>
>> GRAPH <http://www.lingsoft.fi/> {
>> SELECT DISTINCT ?child WHERE {
>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>> }
>> }
>>
>> }
>>
>> Regards,
>> H�vard M. Ottestad
>>
>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>> <mi...@lingsoft.fi> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I'm not if this is the correct forum to ask but hope you can help.
>>> This query takes over 20 seconds with jena:
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>> SELECT DISTINCT ?child WHERE {
>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>> <http://purl.org/dc/terms/isPartOf>
>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>> graph query is for getting keywords from an ontology graph, second
>>> is for querying documents having those keywords. Is there better
>>> way/order to make this query? Thank you for the help, Mikael
>>>
>>> --
>>> www.lingsoft.fi
>>>
>>> Speech Applications - Language Management - Translation - Reader's
>>> and Writer's Tools - Text Tools - E-books and M-books
>>>
>>> Mikael Pesonen
>>> System Engineer
>>>
>>> e-mail: mikael.pesonen@lingsoft.fi
>>> Tel. +358 2 279 3300
>>>
>>> Time zone: GMT+2
>>>
>>> Helsinki Office
>>> Etel�ranta 10
>>> FI-00130 Helsinki
>>> FINLAND
>>>
>>> Turku Office
>>> Linnankatu 10 A
>>> FI-20100 Turku
>>> FINLAND
>>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Query returns ~15k results. I have to get all results since there is
additional filtering done in separate Lucene index afterwards. After
Lucene filter there might be 10 results left.
Also I can't replace ?child with keyword since I have to first get all
children of the selected keywords like this:
<http://www.lingsoft.fi/ontologies/ce3598292f301cec> skos:narrower* ?child
Now came to mind that I could first make the Lucene query, and add that
result as filter to this SPARQL query.
Mikael
On 16.8.2016 15:57, Osma Suominen wrote:
> Hi Mikael!
>
> Okay, so how many results does the query return in those 10 seconds?
> Is it the same if you leave out the DISTINCT? Do you really need all
> those results/triples in a single query result?
>
> Does it help if you replace ?child with an actual keyword? Having an
> unbound variable in there makes the query apply for all subjects
> instead of a specific one.
>
> -Osma
>
> On 16/08/16 15:50, Mikael Pesonen wrote:
>>
>> Hi Osma!
>>
>> unfortunatelly I need to match those certain subjects - the application
>> is document search by selected keywords. And even if
>> ?s <http://purl.org/dc/terms/subject> ?child .
>> is left out the search takes that ~10 seconds.
>>
>> Br,
>> Mikael
>>
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Osma Suominen <os...@helsinki.fi>.
Hi Mikael!
Okay, so how many results does the query return in those 10 seconds? Is
it the same if you leave out the DISTINCT? Do you really need all those
results/triples in a single query result?
Does it help if you replace ?child with an actual keyword? Having an
unbound variable in there makes the query apply for all subjects instead
of a specific one.
-Osma
On 16/08/16 15:50, Mikael Pesonen wrote:
>
> Hi Osma!
>
> unfortunatelly I need to match those certain subjects - the application
> is document search by selected keywords. And even if
> ?s <http://purl.org/dc/terms/subject> ?child .
> is left out the search takes that ~10 seconds.
>
> Br,
> Mikael
>
>
> On 16.8.2016 15:37, Osma Suominen wrote:
>> Hi Mikael,
>>
>> This query will internally create an expensive cross product join if
>> there are many subjects (many different values for ?child), even
>> though the DISTINCT will then mask this internal duplication in the
>> final results.
>>
>> There are many ways to avoid this. I assume that you don't need to
>> know the actual subject values (they are not returned by the query
>> anyway), so you could for example replace ?child with []. Then you
>> also shouldn't need DISTINCT anymore. Like this:
>>
>> SELECT ?s ?p ?o WHERE {
>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>> ?s <http://purl.org/dc/terms/isPartOf>
>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>> ?s <http://purl.org/dc/terms/subject> [] .
>> ?s ?p ?o
>> }
>> }
>>
>> -Osma
>>
>> On 16/08/16 13:36, Mikael Pesonen wrote:
>>>
>>> Inner DISTINCT helps to halve the execution time, but entire query is
>>> low even with that. I also tested without ontology search like this
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>
>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>> ?s <http://purl.org/dc/terms/isPartOf>
>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>> ?s ?p ?o
>>> }
>>>
>>> and it takes almost 10 seconds, so issue is not with the property*
>>> query.
>>>
>>> Is it possible to optimize this basic query or would the only option be
>>> to move the data to RAM from harddrive to get faster queries?
>>>
>>> Mikael
>>>
>>>
>>> On 16.8.2016 13:13, Andy Seaborne wrote:
>>>> On 15/08/16 09:47, Mikael Pesonen wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> what do you mean by masking? It should remove duplicates and it makes
>>>>> the query run in half time compared to without DISTINCT. Result
>>>>> count at
>>>>> least is the same.
>>>>>
>>>>> Mikael
>>>>
>>>> If DISTINCT cause a lot of results to be turned into a few, it is
>>>> hiding a lot of work by the query engine.
>>>>
>>>> If it's the inner DISTINCT that halves the execution time, then the
>>>> improvements (in dev builds) to property* may help you.
>>>>
>>>> If it's the outer one, it's a serialization issue (which I doubt at
>>>> this sacale).
>>>>
>>>> Andy
>>>>
>>>>>
>>>>>
>>>>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>>>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>>>>
>>>>>>> Hi Andy,
>>>>>>>
>>>>>>> storage is started like this:
>>>>>>>
>>>>>>> /usr/bin/java -Xmx3600M -jar
>>>>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>>>>
>>>>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>>>>> metadata triplets. Query returns ~15k triplets.
>>>>>>>
>>>>>>> I tested the SKOS part, and this executed in less than one second,
>>>>>>> returning ~50 items:
>>>>>>
>>>>>> How many without the two DISTINCT?
>>>>>>
>>>>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>>>>> results.
>>>>>>
>>>>>>>
>>>>>>> SELECT DISTINCT *
>>>>>>> WHERE {
>>>>>>> GRAPH ?graph {
>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> UNION
>>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> skos:narrower* ?child}
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> Br,
>>>>>>> Mikael
>>>>>>>
>>>>>>>
>>>>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>>>>
>>>>>>>> In the next release, and available in development builds is:
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>>>>
>>>>>>>> where property* got speeded up recently. Usually, it took
>>>>>>>> moderately
>>>>>>>> unusual data to show this up but the repeated use of an expensive
>>>>>>>> operation in property* may be happening here too.
>>>>>>>>
>>>>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>>>>
>>>>>>>> Andy
>>>>>>>>
>>>>>>>>
>>>>>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>>>>>>> Is this any better?
>>>>>>>>>
>>>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>>>>
>>>>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>>>>> ?s ?p ?o
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>>>>> ?child}
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> H�vard M. Ottestad
>>>>>>>>>
>>>>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I'm not if this is the correct forum to ask but hope you can
>>>>>>>>>> help.
>>>>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>>>>
>>>>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH
>>>>>>>>>> <http://www.lingsoft.fi/> {
>>>>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>>>>>> ?child} }
>>>>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o }
>>>>>>>>>> }First
>>>>>>>>>> graph query is for getting keywords from an ontology graph,
>>>>>>>>>> second
>>>>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> www.lingsoft.fi
>>>>>>>>>>
>>>>>>>>>> Speech Applications - Language Management - Translation -
>>>>>>>>>> Reader's
>>>>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>>>>
>>>>>>>>>> Mikael Pesonen
>>>>>>>>>> System Engineer
>>>>>>>>>>
>>>>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>>>>> Tel. +358 2 279 3300
>>>>>>>>>>
>>>>>>>>>> Time zone: GMT+2
>>>>>>>>>>
>>>>>>>>>> Helsinki Office
>>>>>>>>>> Etel�ranta 10
>>>>>>>>>> FI-00130 Helsinki
>>>>>>>>>> FINLAND
>>>>>>>>>>
>>>>>>>>>> Turku Office
>>>>>>>>>> Linnankatu 10 A
>>>>>>>>>> FI-20100 Turku
>>>>>>>>>> FINLAND
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi Osma!
unfortunatelly I need to match those certain subjects - the application
is document search by selected keywords. And even if
?s <http://purl.org/dc/terms/subject> ?child .
is left out the search takes that ~10 seconds.
Br,
Mikael
On 16.8.2016 15:37, Osma Suominen wrote:
> Hi Mikael,
>
> This query will internally create an expensive cross product join if
> there are many subjects (many different values for ?child), even
> though the DISTINCT will then mask this internal duplication in the
> final results.
>
> There are many ways to avoid this. I assume that you don't need to
> know the actual subject values (they are not returned by the query
> anyway), so you could for example replace ?child with []. Then you
> also shouldn't need DISTINCT anymore. Like this:
>
> SELECT ?s ?p ?o WHERE {
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
> ?s <http://purl.org/dc/terms/isPartOf>
> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
> ?s <http://purl.org/dc/terms/subject> [] .
> ?s ?p ?o
> }
> }
>
> -Osma
>
> On 16/08/16 13:36, Mikael Pesonen wrote:
>>
>> Inner DISTINCT helps to halve the execution time, but entire query is
>> low even with that. I also tested without ontology search like this
>>
>> SELECT DISTINCT ?s ?p ?o WHERE {
>>
>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>> ?s <http://purl.org/dc/terms/isPartOf>
>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>> ?s <http://purl.org/dc/terms/subject> ?child .
>> ?s ?p ?o
>> }
>>
>> and it takes almost 10 seconds, so issue is not with the property*
>> query.
>>
>> Is it possible to optimize this basic query or would the only option be
>> to move the data to RAM from harddrive to get faster queries?
>>
>> Mikael
>>
>>
>> On 16.8.2016 13:13, Andy Seaborne wrote:
>>> On 15/08/16 09:47, Mikael Pesonen wrote:
>>>>
>>>> Hi,
>>>>
>>>> what do you mean by masking? It should remove duplicates and it makes
>>>> the query run in half time compared to without DISTINCT. Result
>>>> count at
>>>> least is the same.
>>>>
>>>> Mikael
>>>
>>> If DISTINCT cause a lot of results to be turned into a few, it is
>>> hiding a lot of work by the query engine.
>>>
>>> If it's the inner DISTINCT that halves the execution time, then the
>>> improvements (in dev builds) to property* may help you.
>>>
>>> If it's the outer one, it's a serialization issue (which I doubt at
>>> this sacale).
>>>
>>> Andy
>>>
>>>>
>>>>
>>>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>>>
>>>>>> Hi Andy,
>>>>>>
>>>>>> storage is started like this:
>>>>>>
>>>>>> /usr/bin/java -Xmx3600M -jar
>>>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>>>
>>>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>>>> metadata triplets. Query returns ~15k triplets.
>>>>>>
>>>>>> I tested the SKOS part, and this executed in less than one second,
>>>>>> returning ~50 items:
>>>>>
>>>>> How many without the two DISTINCT?
>>>>>
>>>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>>>> results.
>>>>>
>>>>>>
>>>>>> SELECT DISTINCT *
>>>>>> WHERE {
>>>>>> GRAPH ?graph {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> UNION
>>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>>>
>>>>>>
>>>>>>
>>>>>> skos:narrower* ?child}
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> Br,
>>>>>> Mikael
>>>>>>
>>>>>>
>>>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>>>
>>>>>>> In the next release, and available in development builds is:
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>>>
>>>>>>> where property* got speeded up recently. Usually, it took
>>>>>>> moderately
>>>>>>> unusual data to show this up but the repeated use of an expensive
>>>>>>> operation in property* may be happening here too.
>>>>>>>
>>>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>>
>>>>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>>>>>> Is this any better?
>>>>>>>>
>>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>>>
>>>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>>>> ?s ?p ?o
>>>>>>>> }
>>>>>>>>
>>>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>>>> ?child}
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> H�vard M. Ottestad
>>>>>>>>
>>>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm not if this is the correct forum to ask but hope you can
>>>>>>>>> help.
>>>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>>>
>>>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH
>>>>>>>>> <http://www.lingsoft.fi/> {
>>>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>>>>> ?child} }
>>>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o }
>>>>>>>>> }First
>>>>>>>>> graph query is for getting keywords from an ontology graph,
>>>>>>>>> second
>>>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> www.lingsoft.fi
>>>>>>>>>
>>>>>>>>> Speech Applications - Language Management - Translation -
>>>>>>>>> Reader's
>>>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>>>
>>>>>>>>> Mikael Pesonen
>>>>>>>>> System Engineer
>>>>>>>>>
>>>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>>>> Tel. +358 2 279 3300
>>>>>>>>>
>>>>>>>>> Time zone: GMT+2
>>>>>>>>>
>>>>>>>>> Helsinki Office
>>>>>>>>> Etel�ranta 10
>>>>>>>>> FI-00130 Helsinki
>>>>>>>>> FINLAND
>>>>>>>>>
>>>>>>>>> Turku Office
>>>>>>>>> Linnankatu 10 A
>>>>>>>>> FI-20100 Turku
>>>>>>>>> FINLAND
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Osma Suominen <os...@helsinki.fi>.
Hi Mikael,
This query will internally create an expensive cross product join if
there are many subjects (many different values for ?child), even though
the DISTINCT will then mask this internal duplication in the final results.
There are many ways to avoid this. I assume that you don't need to know
the actual subject values (they are not returned by the query anyway),
so you could for example replace ?child with []. Then you also shouldn't
need DISTINCT anymore. Like this:
SELECT ?s ?p ?o WHERE {
GRAPH <http://www.lingsoft.fi/resource-meta/> {
?s <http://purl.org/dc/terms/isPartOf>
<http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
?s <http://purl.org/dc/terms/subject> [] .
?s ?p ?o
}
}
-Osma
On 16/08/16 13:36, Mikael Pesonen wrote:
>
> Inner DISTINCT helps to halve the execution time, but entire query is
> low even with that. I also tested without ontology search like this
>
> SELECT DISTINCT ?s ?p ?o WHERE {
>
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
> ?s <http://purl.org/dc/terms/isPartOf>
> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
> ?s <http://purl.org/dc/terms/subject> ?child .
> ?s ?p ?o
> }
>
> and it takes almost 10 seconds, so issue is not with the property* query.
>
> Is it possible to optimize this basic query or would the only option be
> to move the data to RAM from harddrive to get faster queries?
>
> Mikael
>
>
> On 16.8.2016 13:13, Andy Seaborne wrote:
>> On 15/08/16 09:47, Mikael Pesonen wrote:
>>>
>>> Hi,
>>>
>>> what do you mean by masking? It should remove duplicates and it makes
>>> the query run in half time compared to without DISTINCT. Result count at
>>> least is the same.
>>>
>>> Mikael
>>
>> If DISTINCT cause a lot of results to be turned into a few, it is
>> hiding a lot of work by the query engine.
>>
>> If it's the inner DISTINCT that halves the execution time, then the
>> improvements (in dev builds) to property* may help you.
>>
>> If it's the outer one, it's a serialization issue (which I doubt at
>> this sacale).
>>
>> Andy
>>
>>>
>>>
>>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>>
>>>>> Hi Andy,
>>>>>
>>>>> storage is started like this:
>>>>>
>>>>> /usr/bin/java -Xmx3600M -jar
>>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>>
>>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>>> metadata triplets. Query returns ~15k triplets.
>>>>>
>>>>> I tested the SKOS part, and this executed in less than one second,
>>>>> returning ~50 items:
>>>>
>>>> How many without the two DISTINCT?
>>>>
>>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>>> results.
>>>>
>>>>>
>>>>> SELECT DISTINCT *
>>>>> WHERE {
>>>>> GRAPH ?graph {
>>>>> SELECT DISTINCT ?child WHERE {
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> UNION
>>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>>
>>>>>
>>>>> skos:narrower* ?child}
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>> Br,
>>>>> Mikael
>>>>>
>>>>>
>>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>>
>>>>>> In the next release, and available in development builds is:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>>
>>>>>> where property* got speeded up recently. Usually, it took moderately
>>>>>> unusual data to show this up but the repeated use of an expensive
>>>>>> operation in property* may be happening here too.
>>>>>>
>>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>>>>> Is this any better?
>>>>>>>
>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>>
>>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>>> ?s ?p ?o
>>>>>>> }
>>>>>>>
>>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>>> ?child}
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> Regards,
>>>>>>> H�vard M. Ottestad
>>>>>>>
>>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>>
>>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>>> skos:narrower* ?child} UNION
>>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>>>>> graph query is for getting keywords from an ontology graph, second
>>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>>
>>>>>>>> --
>>>>>>>> www.lingsoft.fi
>>>>>>>>
>>>>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>>
>>>>>>>> Mikael Pesonen
>>>>>>>> System Engineer
>>>>>>>>
>>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>>> Tel. +358 2 279 3300
>>>>>>>>
>>>>>>>> Time zone: GMT+2
>>>>>>>>
>>>>>>>> Helsinki Office
>>>>>>>> Etel�ranta 10
>>>>>>>> FI-00130 Helsinki
>>>>>>>> FINLAND
>>>>>>>>
>>>>>>>> Turku Office
>>>>>>>> Linnankatu 10 A
>>>>>>>> FI-20100 Turku
>>>>>>>> FINLAND
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Inner DISTINCT helps to halve the execution time, but entire query is
low even with that. I also tested without ontology search like this
SELECT DISTINCT ?s ?p ?o WHERE {
GRAPH <http://www.lingsoft.fi/resource-meta/> {
?s <http://purl.org/dc/terms/isPartOf>
<http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
?s <http://purl.org/dc/terms/subject> ?child .
?s ?p ?o
}
and it takes almost 10 seconds, so issue is not with the property* query.
Is it possible to optimize this basic query or would the only option be
to move the data to RAM from harddrive to get faster queries?
Mikael
On 16.8.2016 13:13, Andy Seaborne wrote:
> On 15/08/16 09:47, Mikael Pesonen wrote:
>>
>> Hi,
>>
>> what do you mean by masking? It should remove duplicates and it makes
>> the query run in half time compared to without DISTINCT. Result count at
>> least is the same.
>>
>> Mikael
>
> If DISTINCT cause a lot of results to be turned into a few, it is
> hiding a lot of work by the query engine.
>
> If it's the inner DISTINCT that halves the execution time, then the
> improvements (in dev builds) to property* may help you.
>
> If it's the outer one, it's a serialization issue (which I doubt at
> this sacale).
>
> Andy
>
>>
>>
>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>
>>>> Hi Andy,
>>>>
>>>> storage is started like this:
>>>>
>>>> /usr/bin/java -Xmx3600M -jar
>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>
>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>> metadata triplets. Query returns ~15k triplets.
>>>>
>>>> I tested the SKOS part, and this executed in less than one second,
>>>> returning ~50 items:
>>>
>>> How many without the two DISTINCT?
>>>
>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>> results.
>>>
>>>>
>>>> SELECT DISTINCT *
>>>> WHERE {
>>>> GRAPH ?graph {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> }
>>>> }
>>>> }
>>>>
>>>> Br,
>>>> Mikael
>>>>
>>>>
>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>
>>>>> In the next release, and available in development builds is:
>>>>>
>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>
>>>>> where property* got speeded up recently. Usually, it took moderately
>>>>> unusual data to show this up but the repeated use of an expensive
>>>>> operation in property* may be happening here too.
>>>>>
>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>>>> Is this any better?
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>> ?s ?p ?o
>>>>>> }
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>> ?child}
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Regards,
>>>>>> H�vard M. Ottestad
>>>>>>
>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>
>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>>>> graph query is for getting keywords from an ontology graph, second
>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>
>>>>>>> --
>>>>>>> www.lingsoft.fi
>>>>>>>
>>>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>
>>>>>>> Mikael Pesonen
>>>>>>> System Engineer
>>>>>>>
>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>> Tel. +358 2 279 3300
>>>>>>>
>>>>>>> Time zone: GMT+2
>>>>>>>
>>>>>>> Helsinki Office
>>>>>>> Etel�ranta 10
>>>>>>> FI-00130 Helsinki
>>>>>>> FINLAND
>>>>>>>
>>>>>>> Turku Office
>>>>>>> Linnankatu 10 A
>>>>>>> FI-20100 Turku
>>>>>>> FINLAND
>>>>>>>
>>>>>
>>>>
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Leaving child out from inner results was quite obvious mistake. Now
paging works and queries are fast. Thank you!
Mikael
On 4.9.2016 20:31, Andy Seaborne wrote:
>
>
> On 02/09/16 13:22, Mikael Pesonen wrote:
>>
>> Sorry for bombing with questions.
>
> No problem in this case though replies may be delayed - the questions
> aren't always simple things to reply to.
>
> Can you make the data available?
>
>
>> Im trying to add paging with no success:
>>
>> SELECT DISTINCT ?s ?p ?o WHERE {
>> GRAPH ?graph { SELECT DISTINCT ?child WHERE
>> {{<http://www.lingsoft.fi/ontologies/e882a3c73c56c42a> skos:narrower*
>> ?child}}}
>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>
>> { SELECT ?s WHERE {
>>
>> ?s <http://purl.org/dc/terms/subject> ?child .
>> ?s <http://purl.org/dc/terms/isPartOf>
>> <http://www.lingsoft.fi/rdf/uid/57aae39836662> .
>> } limit 2 }
>> ?s ?p ?o .
>> } }
>
> So 2nd the inner SELECT ?s will return 2 items and it hides the
> ?child. You need to add it to the SELECT clause. You might as well
> call it ?Z - the results are same.
>
> And from that block you get rows of ?s ?p ?o. No ?child - it didn't
> get out of the "SELECT ?s"
>
> but the first part "SELECT DISTINCT ?child" has ?child columns - it's
> an unrelated cross product. You'll get all the results for ?child *
> all the triples of 2 isPartOf items.
>
> If trye:
>
> qparse -print=opt you'll see ?child gets renamed to "?/child" in the
> second part.
>
> Andy
>
>> This should return two sets of records having dcterms:subject one of
>> children from ontology query. But I get results where subject seems to
>> be other random items from same ontology.
>>
>>
>> This works, except can't set correct limit:
>>
>> SELECT DISTINCT ?s ?p ?o WHERE {
>> GRAPH ?graph { SELECT DISTINCT ?child WHERE
>> {{<http://www.lingsoft.fi/ontologies/e882a3c73c56c42a> skos:narrower*
>> ?child}}}
>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>
>> ?s <http://purl.org/dc/terms/subject> ?child .
>> ?s <http://purl.org/dc/terms/isPartOf>
>> <http://www.lingsoft.fi/rdf/uid/57aae39836662> .
>>
>> ?s ?p ?o .
>> } }
>> limit 200
>>
>>
>> Mikael
>>
>>
>>
>>
>> On 2.9.2016 12:04, Mikael Pesonen wrote:
>>>
>>> But I think we can handle this by adding paging, so not a show
>>> stopper...
>>>
>>> On 2.9.2016 11:52, Mikael Pesonen wrote:
>>>>
>>>> Tested that one. Example query, similar that Ive sent here, took on
>>>> average ~14 secs on 2.3.1 and 13.5 secs on 2.4.1.
>>>>
>>>> So a bit of improvement but we need to get that query to couple of
>>>> secs so that its usable for our application.
>>>>
>>>> Mikael
>>>>
>>>>
>>>> On 1.9.2016 12:40, Andy Seaborne wrote:
>>>>> On 01/09/16 09:20, Mikael Pesonen wrote:
>>>>>>
>>>>>> How do I get the snapshot? I found this page
>>>>>> https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
>>>>>>
>>>>>> but how to download?
>>>>>>
>>>>>> Mikael
>>>>>
>>>>> The builds end up in a maven repo:
>>>>>
>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>>>>>
>>>>>
>>>>>
>>>>> and you want the "apache-jena-fuseki" module of the SNAPSHOT version:
>>>>>
>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.4.1-SNAPSHOT/
>>>>>
>>>>>
>>>>>
>>>>> Make sure you scroll down: the newest is at the bottom (just at the
>>>>> moment it is "20160831.100731-60"
>>>>>
>>>>> Andy
>>>>>
>>>>
>>>
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi,
On 24.9.2016 17:02, Andy Seaborne wrote:
>
>
> On 23/09/16 09:35, Mikael Pesonen wrote:
>>
>> Hi,
>>
>> I have another query that is behaving illogically to me. I am searching
>> for terms in SKOS vocabulary and also need to retrieve topmost level
>> concept for each search result.
>>
>>
>> This query returns entire skos:broader hierarchy for search results and
>> works in a second (marked related lines with bold)
>>
>> SELECT ?graph ?concept
>> (group_concat(DISTINCT
>> concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
>> ?prefLabelms)
>> (group_concat(DISTINCT
>> concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
>> ?prefLabels)
>> (group_concat(DISTINCT
>> concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS
>> ?altLabelms)
>> (group_concat(DISTINCT
>> concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS
>> ?altLabels)
>> (group_concat(DISTINCT
>> concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
>> (group_concat(DISTINCT
>> concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
>> *(group_concat(DISTINCT
>> concat(?topConceptLabel,"@",lang(?topConceptLabel));separator="/") AS
>> ?topConceptLabels)*
>> WHERE
>> {
>> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
>> GRAPH ?graph
>> {
>> {
>> SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
>> {
>> {?concept skos:prefLabel ?prefLabelm FILTER (
>> (lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
>> REGEX(?prefLabelm, "culo", "i"))}
>> UNION
>> {?concept skos:altLabel ?altLabelm FILTER (
>> (lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
>> REGEX(?altLabelm, "culo", "i"))}
>> }
>> limit 200
>> }
>> ?concept skos:prefLabel ?prefLabel .
>>
>> OPTIONAL { ?concept skos:altLabel ?altLabel }
>> OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
>> rdf:value ?def2 } }
>> *OPTIONAL { ?concept skos:broader* ?topConcept . ?topConcept
>> skos:prefLabel ?topConceptLabel FILTER ( lang(?topConceptLabel) =
>> "fi") }*
>> }
>> }
>> GROUP BY ?graph ?concept
>>
>>
>>
>> But this is what I tried first to get only the one topmost broader for
>> each, but it takes 17 seconds to run:
>>
>> SELECT ?graph ?concept *?topConceptLabel*
>> (group_concat(DISTINCT
>> concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
>> ?prefLabelms)
>> (group_concat(DISTINCT
>> concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
>> ?prefLabels)
>> (group_concat(DISTINCT
>> concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS
>> ?altLabelms)
>> (group_concat(DISTINCT
>> concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS
>> ?altLabels)
>> (group_concat(DISTINCT
>> concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
>> (group_concat(DISTINCT
>> concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
>> WHERE
>> {
>> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
>> GRAPH ?graph
>> {
>> {
>> SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
>> {
>> {?concept skos:prefLabel ?prefLabelm FILTER (
>> (lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
>> REGEX(?prefLabelm, "culo", "i"))}
>> UNION
>> {?concept skos:altLabel ?altLabelm FILTER (
>> (lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
>> REGEX(?altLabelm, "culo", "i"))}
>> }
>> limit 200
>> }
>> ?concept skos:prefLabel ?prefLabel .
>>
>> OPTIONAL { ?concept skos:altLabel ?altLabel }
>> OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
>> rdf:value ?def2 } }
>> *OPTIONAL { ?topConcept skos:topConceptOf ?graph . ?concept
>> skos:broader* ?topConcept . ?topConcept skos:prefLabel ?topConceptLabel
>> FILTER ( lang(?topConceptLabel) = "fi") }*
>
> You can format queries with qparse or use sparql.org.
>
> First:
> GRAPH ?graph
> {
> ...
> OPTIONAL
> { ?concept (skos:broader)* ?topConcept .
> ?topConcept skos:prefLabel ?topConceptLabel
> FILTER ( lang(?topConceptLabel) = "fi" )
> }
>
>
> Second:
> GRAPH ?graph
> {
> ...
> OPTIONAL
> { ?topConcept skos:topConceptOf ?graph .
> ?concept (skos:broader)* ?topConcept .
> ?topConcept skos:prefLabel ?topConceptLabel
> FILTER ( lang(?topConceptLabel) = "fi" )
> }
>
> so the 2nd query does an extra
> "?topConcept skos:topConceptOf ?graph ."
> before the path.
>
> Try putting it after.
Yes, that one reduced time to 10 seconds.
>
> Also,use a version with the path performance fix.
New version was maybe half a second faster.
Okay so maybe SPARQL is not so optimized language yet? With script I can
query all the broader concepts and get the top level one in one sec. Of
course not so elegant and is intuitively more work but gets the job done.
Br,
Mikael
>
> Because you have a group and DISTINCT, this may be several matches but
> this is hidden.
>
> Also, GRAPH ?graph { ... OPTIONAL { use of ?graph .... } ... }
>
> means that the engine may have to separate those two uses of ?graph at
> the point the BGP executes and sort it out later. The earlier
> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
> may not have so much effect limiting the execution search.
>
> Andy
>
>
>
>
>> }
>> }
>> GROUP BY ?graph ?concept *?topConceptLabel*
>>
>>
>>
>> The speed issue seems totally illogical to me but there must be a
>> correct way to perform the latter query then?
>>
>> Br,
>> Mikael
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Okay by trial and error found the solution: move graph selection to
innermost select:
GRAPH ?graph
{
{
SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
{
?graph dcterms:subject "MEDICAL"
{?concept skos:prefLabel ?prefLabelm FILTER (
(lang(?prefLabelm) = "fi" || lang(...
...
Really have to start studying the inner workings of SPARQL queries...
Mikael
On 24.9.2016 17:02, Andy Seaborne wrote:
>
>
> On 23/09/16 09:35, Mikael Pesonen wrote:
>>
>> Hi,
>>
>> I have another query that is behaving illogically to me. I am searching
>> for terms in SKOS vocabulary and also need to retrieve topmost level
>> concept for each search result.
>>
>>
>> This query returns entire skos:broader hierarchy for search results and
>> works in a second (marked related lines with bold)
>>
>> SELECT ?graph ?concept
>> (group_concat(DISTINCT
>> concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
>> ?prefLabelms)
>> (group_concat(DISTINCT
>> concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
>> ?prefLabels)
>> (group_concat(DISTINCT
>> concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS
>> ?altLabelms)
>> (group_concat(DISTINCT
>> concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS
>> ?altLabels)
>> (group_concat(DISTINCT
>> concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
>> (group_concat(DISTINCT
>> concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
>> *(group_concat(DISTINCT
>> concat(?topConceptLabel,"@",lang(?topConceptLabel));separator="/") AS
>> ?topConceptLabels)*
>> WHERE
>> {
>> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
>> GRAPH ?graph
>> {
>> {
>> SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
>> {
>> {?concept skos:prefLabel ?prefLabelm FILTER (
>> (lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
>> REGEX(?prefLabelm, "culo", "i"))}
>> UNION
>> {?concept skos:altLabel ?altLabelm FILTER (
>> (lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
>> REGEX(?altLabelm, "culo", "i"))}
>> }
>> limit 200
>> }
>> ?concept skos:prefLabel ?prefLabel .
>>
>> OPTIONAL { ?concept skos:altLabel ?altLabel }
>> OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
>> rdf:value ?def2 } }
>> *OPTIONAL { ?concept skos:broader* ?topConcept . ?topConcept
>> skos:prefLabel ?topConceptLabel FILTER ( lang(?topConceptLabel) =
>> "fi") }*
>> }
>> }
>> GROUP BY ?graph ?concept
>>
>>
>>
>> But this is what I tried first to get only the one topmost broader for
>> each, but it takes 17 seconds to run:
>>
>> SELECT ?graph ?concept *?topConceptLabel*
>> (group_concat(DISTINCT
>> concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
>> ?prefLabelms)
>> (group_concat(DISTINCT
>> concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
>> ?prefLabels)
>> (group_concat(DISTINCT
>> concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS
>> ?altLabelms)
>> (group_concat(DISTINCT
>> concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS
>> ?altLabels)
>> (group_concat(DISTINCT
>> concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
>> (group_concat(DISTINCT
>> concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
>> WHERE
>> {
>> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
>> GRAPH ?graph
>> {
>> {
>> SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
>> {
>> {?concept skos:prefLabel ?prefLabelm FILTER (
>> (lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
>> REGEX(?prefLabelm, "culo", "i"))}
>> UNION
>> {?concept skos:altLabel ?altLabelm FILTER (
>> (lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
>> REGEX(?altLabelm, "culo", "i"))}
>> }
>> limit 200
>> }
>> ?concept skos:prefLabel ?prefLabel .
>>
>> OPTIONAL { ?concept skos:altLabel ?altLabel }
>> OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
>> rdf:value ?def2 } }
>> *OPTIONAL { ?topConcept skos:topConceptOf ?graph . ?concept
>> skos:broader* ?topConcept . ?topConcept skos:prefLabel ?topConceptLabel
>> FILTER ( lang(?topConceptLabel) = "fi") }*
>
> You can format queries with qparse or use sparql.org.
>
> First:
> GRAPH ?graph
> {
> ...
> OPTIONAL
> { ?concept (skos:broader)* ?topConcept .
> ?topConcept skos:prefLabel ?topConceptLabel
> FILTER ( lang(?topConceptLabel) = "fi" )
> }
>
>
> Second:
> GRAPH ?graph
> {
> ...
> OPTIONAL
> { ?topConcept skos:topConceptOf ?graph .
> ?concept (skos:broader)* ?topConcept .
> ?topConcept skos:prefLabel ?topConceptLabel
> FILTER ( lang(?topConceptLabel) = "fi" )
> }
>
> so the 2nd query does an extra
> "?topConcept skos:topConceptOf ?graph ."
> before the path.
>
> Try putting it after.
>
> Also,use a version with the path performance fix.
>
> Because you have a group and DISTINCT, this may be several matches but
> this is hidden.
>
> Also, GRAPH ?graph { ... OPTIONAL { use of ?graph .... } ... }
>
> means that the engine may have to separate those two uses of ?graph at
> the point the BGP executes and sort it out later. The earlier
> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
> may not have so much effect limiting the execution search.
>
> Andy
>
>
>
>
>> }
>> }
>> GROUP BY ?graph ?concept *?topConceptLabel*
>>
>>
>>
>> The speed issue seems totally illogical to me but there must be a
>> correct way to perform the latter query then?
>>
>> Br,
>> Mikael
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 23/09/16 09:35, Mikael Pesonen wrote:
>
> Hi,
>
> I have another query that is behaving illogically to me. I am searching
> for terms in SKOS vocabulary and also need to retrieve topmost level
> concept for each search result.
>
>
> This query returns entire skos:broader hierarchy for search results and
> works in a second (marked related lines with bold)
>
> SELECT ?graph ?concept
> (group_concat(DISTINCT
> concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
> ?prefLabelms)
> (group_concat(DISTINCT
> concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
> ?prefLabels)
> (group_concat(DISTINCT
> concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS
> ?altLabelms)
> (group_concat(DISTINCT
> concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS ?altLabels)
> (group_concat(DISTINCT
> concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
> (group_concat(DISTINCT
> concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
> *(group_concat(DISTINCT
> concat(?topConceptLabel,"@",lang(?topConceptLabel));separator="/") AS
> ?topConceptLabels)*
> WHERE
> {
> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
> GRAPH ?graph
> {
> {
> SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
> {
> {?concept skos:prefLabel ?prefLabelm FILTER (
> (lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
> REGEX(?prefLabelm, "culo", "i"))}
> UNION
> {?concept skos:altLabel ?altLabelm FILTER (
> (lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
> REGEX(?altLabelm, "culo", "i"))}
> }
> limit 200
> }
> ?concept skos:prefLabel ?prefLabel .
>
> OPTIONAL { ?concept skos:altLabel ?altLabel }
> OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
> rdf:value ?def2 } }
> *OPTIONAL { ?concept skos:broader* ?topConcept . ?topConcept
> skos:prefLabel ?topConceptLabel FILTER ( lang(?topConceptLabel) = "fi") }*
> }
> }
> GROUP BY ?graph ?concept
>
>
>
> But this is what I tried first to get only the one topmost broader for
> each, but it takes 17 seconds to run:
>
> SELECT ?graph ?concept *?topConceptLabel*
> (group_concat(DISTINCT
> concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
> ?prefLabelms)
> (group_concat(DISTINCT
> concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
> ?prefLabels)
> (group_concat(DISTINCT
> concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS
> ?altLabelms)
> (group_concat(DISTINCT
> concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS ?altLabels)
> (group_concat(DISTINCT
> concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
> (group_concat(DISTINCT
> concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
> WHERE
> {
> GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
> GRAPH ?graph
> {
> {
> SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
> {
> {?concept skos:prefLabel ?prefLabelm FILTER (
> (lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
> REGEX(?prefLabelm, "culo", "i"))}
> UNION
> {?concept skos:altLabel ?altLabelm FILTER (
> (lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
> REGEX(?altLabelm, "culo", "i"))}
> }
> limit 200
> }
> ?concept skos:prefLabel ?prefLabel .
>
> OPTIONAL { ?concept skos:altLabel ?altLabel }
> OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
> rdf:value ?def2 } }
> *OPTIONAL { ?topConcept skos:topConceptOf ?graph . ?concept
> skos:broader* ?topConcept . ?topConcept skos:prefLabel ?topConceptLabel
> FILTER ( lang(?topConceptLabel) = "fi") }*
You can format queries with qparse or use sparql.org.
First:
GRAPH ?graph
{
...
OPTIONAL
{ ?concept (skos:broader)* ?topConcept .
?topConcept skos:prefLabel ?topConceptLabel
FILTER ( lang(?topConceptLabel) = "fi" )
}
Second:
GRAPH ?graph
{
...
OPTIONAL
{ ?topConcept skos:topConceptOf ?graph .
?concept (skos:broader)* ?topConcept .
?topConcept skos:prefLabel ?topConceptLabel
FILTER ( lang(?topConceptLabel) = "fi" )
}
so the 2nd query does an extra
"?topConcept skos:topConceptOf ?graph ."
before the path.
Try putting it after.
Also,use a version with the path performance fix.
Because you have a group and DISTINCT, this may be several matches but
this is hidden.
Also, GRAPH ?graph { ... OPTIONAL { use of ?graph .... } ... }
means that the engine may have to separate those two uses of ?graph at
the point the BGP executes and sort it out later. The earlier
GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
may not have so much effect limiting the execution search.
Andy
> }
> }
> GROUP BY ?graph ?concept *?topConceptLabel*
>
>
>
> The speed issue seems totally illogical to me but there must be a
> correct way to perform the latter query then?
>
> Br,
> Mikael
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi,
I have another query that is behaving illogically to me. I am searching
for terms in SKOS vocabulary and also need to retrieve topmost level
concept for each search result.
This query returns entire skos:broader hierarchy for search results and
works in a second (marked related lines with bold)
SELECT ?graph ?concept
(group_concat(DISTINCT
concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
?prefLabelms)
(group_concat(DISTINCT
concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
?prefLabels)
(group_concat(DISTINCT
concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS ?altLabelms)
(group_concat(DISTINCT
concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS ?altLabels)
(group_concat(DISTINCT
concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
(group_concat(DISTINCT
concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
*(group_concat(DISTINCT
concat(?topConceptLabel,"@",lang(?topConceptLabel));separator="/") AS
?topConceptLabels)*
WHERE
{
GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
GRAPH ?graph
{
{
SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
{
{?concept skos:prefLabel ?prefLabelm FILTER (
(lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
REGEX(?prefLabelm, "culo", "i"))}
UNION
{?concept skos:altLabel ?altLabelm FILTER (
(lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
REGEX(?altLabelm, "culo", "i"))}
}
limit 200
}
?concept skos:prefLabel ?prefLabel .
OPTIONAL { ?concept skos:altLabel ?altLabel }
OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
rdf:value ?def2 } }
*OPTIONAL { ?concept skos:broader* ?topConcept . ?topConcept
skos:prefLabel ?topConceptLabel FILTER ( lang(?topConceptLabel) = "fi") }*
}
}
GROUP BY ?graph ?concept
But this is what I tried first to get only the one topmost broader for
each, but it takes 17 seconds to run:
SELECT ?graph ?concept *?topConceptLabel*
(group_concat(DISTINCT
concat(?prefLabelm,"@",lang(?prefLabelm));separator="NEWLINE") AS
?prefLabelms)
(group_concat(DISTINCT
concat(?prefLabel,"@",lang(?prefLabel));separator="NEWLINE") AS
?prefLabels)
(group_concat(DISTINCT
concat(?altLabelm,"@",lang(?altLabelm));separator="NEWLINE") AS ?altLabelms)
(group_concat(DISTINCT
concat(?altLabel,"@",lang(?altLabel));separator="NEWLINE") AS ?altLabels)
(group_concat(DISTINCT
concat(?def1,"@",lang(?def1));separator="NEWLINENEWLINE") AS ?defs1)
(group_concat(DISTINCT
concat(?def2,"@",lang(?def2));separator="NEWLINENEWLINE") AS ?defs2)
WHERE
{
GRAPH ?graph { ?graph dcterms:subject "MEDICAL" }
GRAPH ?graph
{
{
SELECT DISTINCT ?concept ?prefLabelm ?altLabelm WHERE
{
{?concept skos:prefLabel ?prefLabelm FILTER (
(lang(?prefLabelm) = "fi" || lang(?prefLabelm) = "la-FI") &&
REGEX(?prefLabelm, "culo", "i"))}
UNION
{?concept skos:altLabel ?altLabelm FILTER (
(lang(?altLabelm) = "fi" || lang(?altLabelm) = "la-FI") &&
REGEX(?altLabelm, "culo", "i"))}
}
limit 200
}
?concept skos:prefLabel ?prefLabel .
OPTIONAL { ?concept skos:altLabel ?altLabel }
OPTIONAL { ?concept skos:definition ?def1 . OPTIONAL {?def1
rdf:value ?def2 } }
*OPTIONAL { ?topConcept skos:topConceptOf ?graph . ?concept
skos:broader* ?topConcept . ?topConcept skos:prefLabel ?topConceptLabel
FILTER ( lang(?topConceptLabel) = "fi") }*
}
}
GROUP BY ?graph ?concept *?topConceptLabel*
The speed issue seems totally illogical to me but there must be a
correct way to perform the latter query then?
Br,
Mikael
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 02/09/16 13:22, Mikael Pesonen wrote:
>
> Sorry for bombing with questions.
No problem in this case though replies may be delayed - the questions
aren't always simple things to reply to.
Can you make the data available?
> Im trying to add paging with no success:
>
> SELECT DISTINCT ?s ?p ?o WHERE {
> GRAPH ?graph { SELECT DISTINCT ?child WHERE
> {{<http://www.lingsoft.fi/ontologies/e882a3c73c56c42a> skos:narrower*
> ?child}}}
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>
> { SELECT ?s WHERE {
>
> ?s <http://purl.org/dc/terms/subject> ?child .
> ?s <http://purl.org/dc/terms/isPartOf>
> <http://www.lingsoft.fi/rdf/uid/57aae39836662> .
> } limit 2 }
> ?s ?p ?o .
> } }
So 2nd the inner SELECT ?s will return 2 items and it hides the ?child.
You need to add it to the SELECT clause. You might as well call it ?Z -
the results are same.
And from that block you get rows of ?s ?p ?o. No ?child - it didn't get
out of the "SELECT ?s"
but the first part "SELECT DISTINCT ?child" has ?child columns - it's
an unrelated cross product. You'll get all the results for ?child * all
the triples of 2 isPartOf items.
If trye:
qparse -print=opt you'll see ?child gets renamed to "?/child" in the
second part.
Andy
> This should return two sets of records having dcterms:subject one of
> children from ontology query. But I get results where subject seems to
> be other random items from same ontology.
>
>
> This works, except can't set correct limit:
>
> SELECT DISTINCT ?s ?p ?o WHERE {
> GRAPH ?graph { SELECT DISTINCT ?child WHERE
> {{<http://www.lingsoft.fi/ontologies/e882a3c73c56c42a> skos:narrower*
> ?child}}}
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>
> ?s <http://purl.org/dc/terms/subject> ?child .
> ?s <http://purl.org/dc/terms/isPartOf>
> <http://www.lingsoft.fi/rdf/uid/57aae39836662> .
>
> ?s ?p ?o .
> } }
> limit 200
>
>
> Mikael
>
>
>
>
> On 2.9.2016 12:04, Mikael Pesonen wrote:
>>
>> But I think we can handle this by adding paging, so not a show stopper...
>>
>> On 2.9.2016 11:52, Mikael Pesonen wrote:
>>>
>>> Tested that one. Example query, similar that Ive sent here, took on
>>> average ~14 secs on 2.3.1 and 13.5 secs on 2.4.1.
>>>
>>> So a bit of improvement but we need to get that query to couple of
>>> secs so that its usable for our application.
>>>
>>> Mikael
>>>
>>>
>>> On 1.9.2016 12:40, Andy Seaborne wrote:
>>>> On 01/09/16 09:20, Mikael Pesonen wrote:
>>>>>
>>>>> How do I get the snapshot? I found this page
>>>>> https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
>>>>> but how to download?
>>>>>
>>>>> Mikael
>>>>
>>>> The builds end up in a maven repo:
>>>>
>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>>>>
>>>>
>>>> and you want the "apache-jena-fuseki" module of the SNAPSHOT version:
>>>>
>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.4.1-SNAPSHOT/
>>>>
>>>>
>>>> Make sure you scroll down: the newest is at the bottom (just at the
>>>> moment it is "20160831.100731-60"
>>>>
>>>> Andy
>>>>
>>>
>>
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Sorry for bombing with questions. Im trying to add paging with no success:
SELECT DISTINCT ?s ?p ?o WHERE {
GRAPH ?graph { SELECT DISTINCT ?child WHERE
{{<http://www.lingsoft.fi/ontologies/e882a3c73c56c42a> skos:narrower*
?child}}}
GRAPH <http://www.lingsoft.fi/resource-meta/> {
{ SELECT ?s WHERE {
?s <http://purl.org/dc/terms/subject> ?child .
?s <http://purl.org/dc/terms/isPartOf>
<http://www.lingsoft.fi/rdf/uid/57aae39836662> .
} limit 2 }
?s ?p ?o .
} }
This should return two sets of records having dcterms:subject one of
children from ontology query. But I get results where subject seems to
be other random items from same ontology.
This works, except can't set correct limit:
SELECT DISTINCT ?s ?p ?o WHERE {
GRAPH ?graph { SELECT DISTINCT ?child WHERE
{{<http://www.lingsoft.fi/ontologies/e882a3c73c56c42a> skos:narrower*
?child}}}
GRAPH <http://www.lingsoft.fi/resource-meta/> {
?s <http://purl.org/dc/terms/subject> ?child .
?s <http://purl.org/dc/terms/isPartOf>
<http://www.lingsoft.fi/rdf/uid/57aae39836662> .
?s ?p ?o .
} }
limit 200
Mikael
On 2.9.2016 12:04, Mikael Pesonen wrote:
>
> But I think we can handle this by adding paging, so not a show stopper...
>
> On 2.9.2016 11:52, Mikael Pesonen wrote:
>>
>> Tested that one. Example query, similar that Ive sent here, took on
>> average ~14 secs on 2.3.1 and 13.5 secs on 2.4.1.
>>
>> So a bit of improvement but we need to get that query to couple of
>> secs so that its usable for our application.
>>
>> Mikael
>>
>>
>> On 1.9.2016 12:40, Andy Seaborne wrote:
>>> On 01/09/16 09:20, Mikael Pesonen wrote:
>>>>
>>>> How do I get the snapshot? I found this page
>>>> https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
>>>> but how to download?
>>>>
>>>> Mikael
>>>
>>> The builds end up in a maven repo:
>>>
>>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>>>
>>>
>>> and you want the "apache-jena-fuseki" module of the SNAPSHOT version:
>>>
>>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.4.1-SNAPSHOT/
>>>
>>>
>>> Make sure you scroll down: the newest is at the bottom (just at the
>>> moment it is "20160831.100731-60"
>>>
>>> Andy
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
But I think we can handle this by adding paging, so not a show stopper...
On 2.9.2016 11:52, Mikael Pesonen wrote:
>
> Tested that one. Example query, similar that Ive sent here, took on
> average ~14 secs on 2.3.1 and 13.5 secs on 2.4.1.
>
> So a bit of improvement but we need to get that query to couple of
> secs so that its usable for our application.
>
> Mikael
>
>
> On 1.9.2016 12:40, Andy Seaborne wrote:
>> On 01/09/16 09:20, Mikael Pesonen wrote:
>>>
>>> How do I get the snapshot? I found this page
>>> https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
>>> but how to download?
>>>
>>> Mikael
>>
>> The builds end up in a maven repo:
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>>
>>
>> and you want the "apache-jena-fuseki" module of the SNAPSHOT version:
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.4.1-SNAPSHOT/
>>
>>
>> Make sure you scroll down: the newest is at the bottom (just at the
>> moment it is "20160831.100731-60"
>>
>> Andy
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Tested that one. Example query, similar that Ive sent here, took on
average ~14 secs on 2.3.1 and 13.5 secs on 2.4.1.
So a bit of improvement but we need to get that query to couple of secs
so that its usable for our application.
Mikael
On 1.9.2016 12:40, Andy Seaborne wrote:
> On 01/09/16 09:20, Mikael Pesonen wrote:
>>
>> How do I get the snapshot? I found this page
>> https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
>> but how to download?
>>
>> Mikael
>
> The builds end up in a maven repo:
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
>
>
> and you want the "apache-jena-fuseki" module of the SNAPSHOT version:
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.4.1-SNAPSHOT/
>
>
> Make sure you scroll down: the newest is at the bottom (just at the
> moment it is "20160831.100731-60"
>
> Andy
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 01/09/16 09:20, Mikael Pesonen wrote:
>
> How do I get the snapshot? I found this page
> https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
> but how to download?
>
> Mikael
The builds end up in a maven repo:
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/
and you want the "apache-jena-fuseki" module of the SNAPSHOT version:
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.4.1-SNAPSHOT/
Make sure you scroll down: the newest is at the bottom (just at the
moment it is "20160831.100731-60"
Andy
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
How do I get the snapshot? I found this page
https://builds.apache.org/job/Jena_Development_Deploy/lastStableBuild/
but how to download?
Mikael
On 28.8.2016 13:34, Andy Seaborne wrote:
> On 26/08/16 12:17, Mikael Pesonen wrote:
>>
>> I'm happy to try out the snapshot. Its just matter of running the server
>> - no modifications of data or config needed?
>>
>> Do you know when the new version will released (weeks, months)?
>>
>> Mikael
>
> We've started talking about it:
>
> http://mail-archives.apache.org/mod_mbox/jena-dev/201608.mbox/%3C6c93affa-8a09-1d87-2ef2-b235b7a1a8a5%40apache.org%3E
>
>
> (open to everyone)
>
> We can't commit to dates because volunteer time ebbs and flows. Other
> things can overtake plans so we can't reliably commit to anything.
> But we aim for once every 6 months and the last release was May and I
> think we could go a bit earlier this time.
>
> There is some outstanding items that are desirable to deal with, but
> not blockers.
>
> Having reports of good (preferably!) and bad experiences with the
> development builds (get from the snapshot repository - they are simply
> the state of the codebase at the time of the build - done once a day).
>
> Andy
>
>>
>>
>> On 26.8.2016 13:53, Andy Seaborne wrote:
>>> On 26/08/16 11:35, Rob Vesse wrote:
>>>> To try to answer the question about your specific query it\u2019s
>>>> difficult without knowing more about the nature of the data, in your
>>>> case how many named graphs are in the database?
>>>>
>>>> One thing that jumps out at me is that you use the GRAPH operator in
>>>> your query. That operator essentially requires that a query engine
>>>> applies your inner query to every single graph in your database. In
>>>> practice ARQ Will try to do something a bit more efficient than that
>>>> but this is not guaranteed.
>>>
>>> TDB does all the graphs at the same time if it can. Property paths can
>>> stop this but basic graph patterns + GRAPH ?var is done as quad table
>>> accesses.
>>>
>>> Ditto for TIM.
>>>
>>> (It's only the general purpose in-memory dataset where you can put any
>>> graph in from any source that has to loop.)
>>>
>>>> Your inner query uses a lots of property paths and so is potentially
>>>> quite expensive. As a first step I would suggest changing * to + if
>>>> you can as that will avoid having to match the Zero length path which
>>>> while quite simple for your case can be very expensive for more
>>>> generic property paths.
>>>
>>> And has been speed up recently (after the last release I'm afraid,
>>> post Jena 3.1.0 (Fuseki 2.4.0)) JENA-1195
>>>
>>> How long are the skos:narrower* chains?
>>>
>>> Mikael - are you able to try out a SNAPSHOT build?
>>>
>>>> Secondly if you are able to limit the number of graphs that are under
>>>> consideration you may be able to substantially improved performance.
>>>> One way to do this would be to place a pattern prior to the GRAPH
>>>> operator that restricts the values of the ?graph variable. ARQ
>>>> should then be able to use that information to restrict which graphs
>>>> in the database it scans.
>>>>
>>>> Rob
>>>>
>>>
>>> Andy
>>>
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 26/08/16 12:17, Mikael Pesonen wrote:
>
> I'm happy to try out the snapshot. Its just matter of running the server
> - no modifications of data or config needed?
>
> Do you know when the new version will released (weeks, months)?
>
> Mikael
We've started talking about it:
http://mail-archives.apache.org/mod_mbox/jena-dev/201608.mbox/%3C6c93affa-8a09-1d87-2ef2-b235b7a1a8a5%40apache.org%3E
(open to everyone)
We can't commit to dates because volunteer time ebbs and flows. Other
things can overtake plans so we can't reliably commit to anything. But
we aim for once every 6 months and the last release was May and I think
we could go a bit earlier this time.
There is some outstanding items that are desirable to deal with, but not
blockers.
Having reports of good (preferably!) and bad experiences with the
development builds (get from the snapshot repository - they are simply
the state of the codebase at the time of the build - done once a day).
Andy
>
>
> On 26.8.2016 13:53, Andy Seaborne wrote:
>> On 26/08/16 11:35, Rob Vesse wrote:
>>> To try to answer the question about your specific query it\u2019s
>>> difficult without knowing more about the nature of the data, in your
>>> case how many named graphs are in the database?
>>>
>>> One thing that jumps out at me is that you use the GRAPH operator in
>>> your query. That operator essentially requires that a query engine
>>> applies your inner query to every single graph in your database. In
>>> practice ARQ Will try to do something a bit more efficient than that
>>> but this is not guaranteed.
>>
>> TDB does all the graphs at the same time if it can. Property paths can
>> stop this but basic graph patterns + GRAPH ?var is done as quad table
>> accesses.
>>
>> Ditto for TIM.
>>
>> (It's only the general purpose in-memory dataset where you can put any
>> graph in from any source that has to loop.)
>>
>>> Your inner query uses a lots of property paths and so is potentially
>>> quite expensive. As a first step I would suggest changing * to + if
>>> you can as that will avoid having to match the Zero length path which
>>> while quite simple for your case can be very expensive for more
>>> generic property paths.
>>
>> And has been speed up recently (after the last release I'm afraid,
>> post Jena 3.1.0 (Fuseki 2.4.0)) JENA-1195
>>
>> How long are the skos:narrower* chains?
>>
>> Mikael - are you able to try out a SNAPSHOT build?
>>
>>> Secondly if you are able to limit the number of graphs that are under
>>> consideration you may be able to substantially improved performance.
>>> One way to do this would be to place a pattern prior to the GRAPH
>>> operator that restricts the values of the ?graph variable. ARQ
>>> should then be able to use that information to restrict which graphs
>>> in the database it scans.
>>>
>>> Rob
>>>
>>
>> Andy
>>
>>
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
I'm happy to try out the snapshot. Its just matter of running the server
- no modifications of data or config needed?
Do you know when the new version will released (weeks, months)?
Mikael
On 26.8.2016 13:53, Andy Seaborne wrote:
> On 26/08/16 11:35, Rob Vesse wrote:
>> To try to answer the question about your specific query it\u2019s
>> difficult without knowing more about the nature of the data, in your
>> case how many named graphs are in the database?
>>
>> One thing that jumps out at me is that you use the GRAPH operator in
>> your query. That operator essentially requires that a query engine
>> applies your inner query to every single graph in your database. In
>> practice ARQ Will try to do something a bit more efficient than that
>> but this is not guaranteed.
>
> TDB does all the graphs at the same time if it can. Property paths can
> stop this but basic graph patterns + GRAPH ?var is done as quad table
> accesses.
>
> Ditto for TIM.
>
> (It's only the general purpose in-memory dataset where you can put any
> graph in from any source that has to loop.)
>
>> Your inner query uses a lots of property paths and so is potentially
>> quite expensive. As a first step I would suggest changing * to + if
>> you can as that will avoid having to match the Zero length path which
>> while quite simple for your case can be very expensive for more
>> generic property paths.
>
> And has been speed up recently (after the last release I'm afraid,
> post Jena 3.1.0 (Fuseki 2.4.0)) JENA-1195
>
> How long are the skos:narrower* chains?
>
> Mikael - are you able to try out a SNAPSHOT build?
>
>> Secondly if you are able to limit the number of graphs that are under
>> consideration you may be able to substantially improved performance.
>> One way to do this would be to place a pattern prior to the GRAPH
>> operator that restricts the values of the ?graph variable. ARQ
>> should then be able to use that information to restrict which graphs
>> in the database it scans.
>>
>> Rob
>>
>
> Andy
>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 26/08/16 11:35, Rob Vesse wrote:
> To try to answer the question about your specific query it\u2019s
> difficult without knowing more about the nature of the data, in your
> case how many named graphs are in the database?
>
> One thing that jumps out at me is that you use the GRAPH operator in
> your query. That operator essentially requires that a query engine
> applies your inner query to every single graph in your database. In
> practice ARQ Will try to do something a bit more efficient than that
> but this is not guaranteed.
TDB does all the graphs at the same time if it can. Property paths can
stop this but basic graph patterns + GRAPH ?var is done as quad table
accesses.
Ditto for TIM.
(It's only the general purpose in-memory dataset where you can put any
graph in from any source that has to loop.)
> Your inner query uses a lots of property paths and so is potentially
> quite expensive. As a first step I would suggest changing * to + if
> you can as that will avoid having to match the Zero length path which
> while quite simple for your case can be very expensive for more
> generic property paths.
And has been speed up recently (after the last release I'm afraid, post
Jena 3.1.0 (Fuseki 2.4.0)) JENA-1195
How long are the skos:narrower* chains?
Mikael - are you able to try out a SNAPSHOT build?
> Secondly if you are able to limit the number of graphs that are under
> consideration you may be able to substantially improved performance.
> One way to do this would be to place a pattern prior to the GRAPH
> operator that restricts the values of the ?graph variable. ARQ
> should then be able to use that information to restrict which graphs
> in the database it scans.
>
> Rob
>
Andy
Re: Slow SPARQL query
Posted by Rob Vesse <rv...@dotnetrdf.org>.
To try to answer the question about your specific query it’s difficult without knowing more about the nature of the data, in your case how many named graphs are in the database?
One thing that jumps out at me is that you use the GRAPH operator in your query. That operator essentially requires that a query engine applies your inner query to every single graph in your database. In practice ARQ Will try to do something a bit more efficient than that but this is not guaranteed.
Your inner query uses a lots of property paths and so is potentially quite expensive. As a first step I would suggest changing * to + if you can as that will avoid having to match the Zero length path which while quite simple for your case can be very expensive for more generic property paths.
Secondly if you are able to limit the number of graphs that are under consideration you may be able to substantially improved performance. One way to do this would be to place a pattern prior to the GRAPH operator that restricts the values of the ?graph variable. ARQ should then be able to use that information to restrict which graphs in the database it scans.
Rob
On 26/08/2016 10:46, "Mikael Pesonen" <mi...@lingsoft.fi> wrote:
Hi, still wondering what I should do to make the performance better.
I read that TDB is faster. What is the reason not to use TDB? Cant find
any comparison on SDB and TDB in that regard.
Br,
Mikael
On 16.8.2016 13:13, Andy Seaborne wrote:
> On 15/08/16 09:47, Mikael Pesonen wrote:
>>
>> Hi,
>>
>> what do you mean by masking? It should remove duplicates and it makes
>> the query run in half time compared to without DISTINCT. Result count at
>> least is the same.
>>
>> Mikael
>
> If DISTINCT cause a lot of results to be turned into a few, it is
> hiding a lot of work by the query engine.
>
> If it's the inner DISTINCT that halves the execution time, then the
> improvements (in dev builds) to property* may help you.
>
> If it's the outer one, it's a serialization issue (which I doubt at
> this sacale).
>
> Andy
>
>>
>>
>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>
>>>> Hi Andy,
>>>>
>>>> storage is started like this:
>>>>
>>>> /usr/bin/java -Xmx3600M -jar
>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>
>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>> metadata triplets. Query returns ~15k triplets.
>>>>
>>>> I tested the SKOS part, and this executed in less than one second,
>>>> returning ~50 items:
>>>
>>> How many without the two DISTINCT?
>>>
>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>> results.
>>>
>>>>
>>>> SELECT DISTINCT *
>>>> WHERE {
>>>> GRAPH ?graph {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> }
>>>> }
>>>> }
>>>>
>>>> Br,
>>>> Mikael
>>>>
>>>>
>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>
>>>>> In the next release, and available in development builds is:
>>>>>
>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>
>>>>> where property* got speeded up recently. Usually, it took moderately
>>>>> unusual data to show this up but the repeated use of an expensive
>>>>> operation in property* may be happening here too.
>>>>>
>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>>>>> Is this any better?
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>> ?s ?p ?o
>>>>>> }
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>> ?child}
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Regards,
>>>>>> Håvard M. Ottestad
>>>>>>
>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>
>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>>>> graph query is for getting keywords from an ontology graph, second
>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>
>>>>>>> --
>>>>>>> www.lingsoft.fi
>>>>>>>
>>>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>
>>>>>>> Mikael Pesonen
>>>>>>> System Engineer
>>>>>>>
>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>> Tel. +358 2 279 3300
>>>>>>>
>>>>>>> Time zone: GMT+2
>>>>>>>
>>>>>>> Helsinki Office
>>>>>>> Eteläranta 10
>>>>>>> FI-00130 Helsinki
>>>>>>> FINLAND
>>>>>>>
>>>>>>> Turku Office
>>>>>>> Linnankatu 10 A
>>>>>>> FI-20100 Turku
>>>>>>> FINLAND
>>>>>>>
>>>>>
>>>>
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Haven't done any changes there :-)
-Mikael
On 26.8.2016 13:55, A. Soroka wrote:
> Yes, unless you have made some changes in configuration that you would not have made without noticing. {grin} TDB is the default storage for Fuseki.
>
> ---
> A. Soroka
> The University of Virginia Library
>
>> On Aug 26, 2016, at 6:40 AM, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>
>>
>> Hi Rob,
>>
>> Im using this command to start db:
>>
>> /usr/bin/java -Xmx3600M -jar apache-jena-fuseki-2.3.1/fuseki-server.jar --update --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>
>> and s- command line tools to make queries. In documentation there is tdbquery which I'm not using. But TDB is still in use?
>>
>> Thanks,
>> Mikael
>>
>>
>>
>> On 26.8.2016 13:27, Rob Vesse wrote:
>>> Mikael
>>>
>>> If you\u2019re using Fuseki then you are using TDB already. TDB is a native RDF database that uses memory mapped files to make data access as fast as possible.
>>>
>>> SDB is a legacy system built on top of relational databases, so queries have to be compiled into SQL, submitted to the underlying relational database, and their results translated back into RDF appropriately. More complex queries cannot be translated directly into a single SQL query due to the differing semantics between the two query languages I may require many SQL Queries to answer. SDB is no longer actively developed and receives only minor bug fixes.
>>>
>>> As for when you would not use TDB no probably three main criteria:
>>>
>>> 1 - the amount of data you will store is that into the billions of triples. TDB Will scale pretty well into the millions of triples although this will depend on the complexity of the queries.
>>> 2 - when you need clustering for load balancing, failover etc. TDB is a single node system, while there are ways to do load balancing these typically rely on layering additional services on top of it
>>> 3 - when you need reasoning support. TDB does not natively support reasoning, you can use other Jena apis to add this but they Will substantially be great performance because they require all the data to be in heap memory. If your data is static then you can compute the inference closure once and persist that into the database but if you need dynamic inference or extremely large-scale inference then TDB Will not be suitable.
>>>
>>> There are plenty of commercial options that do address the above three criteria and people can probably provide recommendations if you think you need a commercial option.
>>>
>>> It is also worth noting that some queries are simply hard for any query engine to answer
>>>
>>> Rob
>>>
>>> On 26/08/2016 10:46, "Mikael Pesonen" <mi...@lingsoft.fi> wrote:
>>>
>>> Hi, still wondering what I should do to make the performance better.
>>> I read that TDB is faster. What is the reason not to use TDB? Cant find
>>> any comparison on SDB and TDB in that regard.
>>> Br,
>>> Mikael
>>> On 16.8.2016 13:13, Andy Seaborne wrote:
>>> > On 15/08/16 09:47, Mikael Pesonen wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> what do you mean by masking? It should remove duplicates and it makes
>>> >> the query run in half time compared to without DISTINCT. Result count at
>>> >> least is the same.
>>> >>
>>> >> Mikael
>>> >
>>> > If DISTINCT cause a lot of results to be turned into a few, it is
>>> > hiding a lot of work by the query engine.
>>> >
>>> > If it's the inner DISTINCT that halves the execution time, then the
>>> > improvements (in dev builds) to property* may help you.
>>> >
>>> > If it's the outer one, it's a serialization issue (which I doubt at
>>> > this sacale).
>>> >
>>> > Andy
>>> >
>>> >>
>>> >>
>>> >> On 12.8.2016 13:53, Andy Seaborne wrote:
>>> >>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>> >>>>
>>> >>>> Hi Andy,
>>> >>>>
>>> >>>> storage is started like this:
>>> >>>>
>>> >>>> /usr/bin/java -Xmx3600M -jar
>>> >>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>> >>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>> >>>>
>>> >>>> Ontology data is simple SKOS, and document data is also simple DC
>>> >>>> metadata triplets. Query returns ~15k triplets.
>>> >>>>
>>> >>>> I tested the SKOS part, and this executed in less than one second,
>>> >>>> returning ~50 items:
>>> >>>
>>> >>> How many without the two DISTINCT?
>>> >>>
>>> >>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>> >>> results.
>>> >>>
>>> >>>>
>>> >>>> SELECT DISTINCT *
>>> >>>> WHERE {
>>> >>>> GRAPH ?graph {
>>> >>>> SELECT DISTINCT ?child WHERE {
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> UNION
>>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>> >>>>
>>> >>>>
>>> >>>> skos:narrower* ?child}
>>> >>>> }
>>> >>>> }
>>> >>>> }
>>> >>>>
>>> >>>> Br,
>>> >>>> Mikael
>>> >>>>
>>> >>>>
>>> >>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>> >>>>> There is a a certain amount of "it depend" here: what's the data
>>> >>>>> stored in? what shape is the data?( which Jena version?)
>>> >>>>>
>>> >>>>> In the next release, and available in development builds is:
>>> >>>>>
>>> >>>>> https://issues.apache.org/jira/browse/JENA-1195
>>> >>>>>
>>> >>>>> where property* got speeded up recently. Usually, it took moderately
>>> >>>>> unusual data to show this up but the repeated use of an expensive
>>> >>>>> operation in property* may be happening here too.
>>> >>>>>
>>> >>>>> Mikael - are you able to try out a SNAPSHOT build?
>>> >>>>>
>>> >>>>> Andy
>>> >>>>>
>>> >>>>>
>>> >>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>> >>>>>> Is this any better?
>>> >>>>>>
>>> >>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>> >>>>>>
>>> >>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>> >>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>> >>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>> >>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>> >>>>>> ?s ?p ?o
>>> >>>>>> }
>>> >>>>>>
>>> >>>>>> GRAPH <http://www.lingsoft.fi/> {
>>> >>>>>> SELECT DISTINCT ?child WHERE {
>>> >>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>> >>>>>> skos:narrower* ?child} UNION
>>> >>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>> >>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>> >>>>>> skos:narrower* ?child} UNION
>>> >>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>> >>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>> >>>>>> skos:narrower* ?child} UNION
>>> >>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>> >>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>> >>>>>> skos:narrower* ?child} UNION
>>> >>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>> >>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>> >>>>>> ?child}
>>> >>>>>> }
>>> >>>>>> }
>>> >>>>>>
>>> >>>>>> }
>>> >>>>>>
>>> >>>>>> Regards,
>>> >>>>>> H�vard M. Ottestad
>>> >>>>>>
>>> >>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>> >>>>>>> <mi...@lingsoft.fi> wrote:
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Hi,
>>> >>>>>>>
>>> >>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>> >>>>>>> This query takes over 20 seconds with jena:
>>> >>>>>>>
>>> >>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>> >>>>>>> SELECT DISTINCT ?child WHERE {
>>> >>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>> >>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>> >>>>>>> skos:narrower* ?child} UNION
>>> >>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>> >>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>> >>>>>>> skos:narrower* ?child} UNION
>>> >>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>> >>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>> >>>>>>> skos:narrower* ?child} UNION
>>> >>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>> >>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>> >>>>>>> skos:narrower* ?child} UNION
>>> >>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>> >>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>> >>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>> >>>>>>> <http://purl.org/dc/terms/isPartOf>
>>> >>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>> >>>>>>> graph query is for getting keywords from an ontology graph, second
>>> >>>>>>> is for querying documents having those keywords. Is there better
>>> >>>>>>> way/order to make this query? Thank you for the help, Mikael
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> www.lingsoft.fi
>>> >>>>>>>
>>> >>>>>>> Speech Applications - Language Management - Translation - Reader's
>>> >>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>> >>>>>>>
>>> >>>>>>> Mikael Pesonen
>>> >>>>>>> System Engineer
>>> >>>>>>>
>>> >>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>> >>>>>>> Tel. +358 2 279 3300
>>> >>>>>>>
>>> >>>>>>> Time zone: GMT+2
>>> >>>>>>>
>>> >>>>>>> Helsinki Office
>>> >>>>>>> Etel�ranta 10
>>> >>>>>>> FI-00130 Helsinki
>>> >>>>>>> FINLAND
>>> >>>>>>>
>>> >>>>>>> Turku Office
>>> >>>>>>> Linnankatu 10 A
>>> >>>>>>> FI-20100 Turku
>>> >>>>>>> FINLAND
>>> >>>>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >
>>> --
>>> www.lingsoft.fi
>>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>> Mikael Pesonen
>>> System Engineer
>>> e-mail: mikael.pesonen@lingsoft.fi
>>> Tel. +358 2 279 3300
>>> Time zone: GMT+2
>>> Helsinki Office
>>> Etel�ranta 10
>>> FI-00130 Helsinki
>>> FINLAND
>>> Turku Office
>>> Linnankatu 10 A
>>> FI-20100 Turku
>>> FINLAND
>>>
>>>
>>>
>>>
>> --
>> www.lingsoft.fi
>>
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>
>> Mikael Pesonen
>> System Engineer
>>
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>>
>> Time zone: GMT+2
>>
>> Helsinki Office
>> Etel�ranta 10
>> FI-00130 Helsinki
>> FINLAND
>>
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by "A. Soroka" <aj...@virginia.edu>.
Yes, unless you have made some changes in configuration that you would not have made without noticing. {grin} TDB is the default storage for Fuseki.
---
A. Soroka
The University of Virginia Library
> On Aug 26, 2016, at 6:40 AM, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>
>
> Hi Rob,
>
> Im using this command to start db:
>
> /usr/bin/java -Xmx3600M -jar apache-jena-fuseki-2.3.1/fuseki-server.jar --update --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>
> and s- command line tools to make queries. In documentation there is tdbquery which I'm not using. But TDB is still in use?
>
> Thanks,
> Mikael
>
>
>
> On 26.8.2016 13:27, Rob Vesse wrote:
>> Mikael
>>
>> If you’re using Fuseki then you are using TDB already. TDB is a native RDF database that uses memory mapped files to make data access as fast as possible.
>>
>> SDB is a legacy system built on top of relational databases, so queries have to be compiled into SQL, submitted to the underlying relational database, and their results translated back into RDF appropriately. More complex queries cannot be translated directly into a single SQL query due to the differing semantics between the two query languages I may require many SQL Queries to answer. SDB is no longer actively developed and receives only minor bug fixes.
>>
>> As for when you would not use TDB no probably three main criteria:
>>
>> 1 - the amount of data you will store is that into the billions of triples. TDB Will scale pretty well into the millions of triples although this will depend on the complexity of the queries.
>> 2 - when you need clustering for load balancing, failover etc. TDB is a single node system, while there are ways to do load balancing these typically rely on layering additional services on top of it
>> 3 - when you need reasoning support. TDB does not natively support reasoning, you can use other Jena apis to add this but they Will substantially be great performance because they require all the data to be in heap memory. If your data is static then you can compute the inference closure once and persist that into the database but if you need dynamic inference or extremely large-scale inference then TDB Will not be suitable.
>>
>> There are plenty of commercial options that do address the above three criteria and people can probably provide recommendations if you think you need a commercial option.
>>
>> It is also worth noting that some queries are simply hard for any query engine to answer
>>
>> Rob
>>
>> On 26/08/2016 10:46, "Mikael Pesonen" <mi...@lingsoft.fi> wrote:
>>
>> Hi, still wondering what I should do to make the performance better.
>> I read that TDB is faster. What is the reason not to use TDB? Cant find
>> any comparison on SDB and TDB in that regard.
>> Br,
>> Mikael
>> On 16.8.2016 13:13, Andy Seaborne wrote:
>> > On 15/08/16 09:47, Mikael Pesonen wrote:
>> >>
>> >> Hi,
>> >>
>> >> what do you mean by masking? It should remove duplicates and it makes
>> >> the query run in half time compared to without DISTINCT. Result count at
>> >> least is the same.
>> >>
>> >> Mikael
>> >
>> > If DISTINCT cause a lot of results to be turned into a few, it is
>> > hiding a lot of work by the query engine.
>> >
>> > If it's the inner DISTINCT that halves the execution time, then the
>> > improvements (in dev builds) to property* may help you.
>> >
>> > If it's the outer one, it's a serialization issue (which I doubt at
>> > this sacale).
>> >
>> > Andy
>> >
>> >>
>> >>
>> >> On 12.8.2016 13:53, Andy Seaborne wrote:
>> >>> On 08/08/16 11:56, Mikael Pesonen wrote:
>> >>>>
>> >>>> Hi Andy,
>> >>>>
>> >>>> storage is started like this:
>> >>>>
>> >>>> /usr/bin/java -Xmx3600M -jar
>> >>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>> >>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>> >>>>
>> >>>> Ontology data is simple SKOS, and document data is also simple DC
>> >>>> metadata triplets. Query returns ~15k triplets.
>> >>>>
>> >>>> I tested the SKOS part, and this executed in less than one second,
>> >>>> returning ~50 items:
>> >>>
>> >>> How many without the two DISTINCT?
>> >>>
>> >>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>> >>> results.
>> >>>
>> >>>>
>> >>>> SELECT DISTINCT *
>> >>>> WHERE {
>> >>>> GRAPH ?graph {
>> >>>> SELECT DISTINCT ?child WHERE {
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> UNION
>> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>> >>>>
>> >>>>
>> >>>> skos:narrower* ?child}
>> >>>> }
>> >>>> }
>> >>>> }
>> >>>>
>> >>>> Br,
>> >>>> Mikael
>> >>>>
>> >>>>
>> >>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>> >>>>> There is a a certain amount of "it depend" here: what's the data
>> >>>>> stored in? what shape is the data?( which Jena version?)
>> >>>>>
>> >>>>> In the next release, and available in development builds is:
>> >>>>>
>> >>>>> https://issues.apache.org/jira/browse/JENA-1195
>> >>>>>
>> >>>>> where property* got speeded up recently. Usually, it took moderately
>> >>>>> unusual data to show this up but the repeated use of an expensive
>> >>>>> operation in property* may be happening here too.
>> >>>>>
>> >>>>> Mikael - are you able to try out a SNAPSHOT build?
>> >>>>>
>> >>>>> Andy
>> >>>>>
>> >>>>>
>> >>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>> >>>>>> Is this any better?
>> >>>>>>
>> >>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>> >>>>>>
>> >>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>> >>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>> >>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>> >>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>> >>>>>> ?s ?p ?o
>> >>>>>> }
>> >>>>>>
>> >>>>>> GRAPH <http://www.lingsoft.fi/> {
>> >>>>>> SELECT DISTINCT ?child WHERE {
>> >>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>> >>>>>> skos:narrower* ?child} UNION
>> >>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>> >>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>> >>>>>> skos:narrower* ?child} UNION
>> >>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>> >>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>> >>>>>> skos:narrower* ?child} UNION
>> >>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>> >>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>> >>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>> >>>>>> skos:narrower* ?child} UNION
>> >>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>> >>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>> >>>>>> ?child}
>> >>>>>> }
>> >>>>>> }
>> >>>>>>
>> >>>>>> }
>> >>>>>>
>> >>>>>> Regards,
>> >>>>>> Håvard M. Ottestad
>> >>>>>>
>> >>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>> >>>>>>> <mi...@lingsoft.fi> wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi,
>> >>>>>>>
>> >>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>> >>>>>>> This query takes over 20 seconds with jena:
>> >>>>>>>
>> >>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>> >>>>>>> SELECT DISTINCT ?child WHERE {
>> >>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>> >>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>> >>>>>>> skos:narrower* ?child} UNION
>> >>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>> >>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>> >>>>>>> skos:narrower* ?child} UNION
>> >>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>> >>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>> >>>>>>> skos:narrower* ?child} UNION
>> >>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>> >>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>> >>>>>>> skos:narrower* ?child} UNION
>> >>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>> >>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>> >>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>> >>>>>>> <http://purl.org/dc/terms/isPartOf>
>> >>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>> >>>>>>> graph query is for getting keywords from an ontology graph, second
>> >>>>>>> is for querying documents having those keywords. Is there better
>> >>>>>>> way/order to make this query? Thank you for the help, Mikael
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> www.lingsoft.fi
>> >>>>>>>
>> >>>>>>> Speech Applications - Language Management - Translation - Reader's
>> >>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>> >>>>>>>
>> >>>>>>> Mikael Pesonen
>> >>>>>>> System Engineer
>> >>>>>>>
>> >>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>> >>>>>>> Tel. +358 2 279 3300
>> >>>>>>>
>> >>>>>>> Time zone: GMT+2
>> >>>>>>>
>> >>>>>>> Helsinki Office
>> >>>>>>> Eteläranta 10
>> >>>>>>> FI-00130 Helsinki
>> >>>>>>> FINLAND
>> >>>>>>>
>> >>>>>>> Turku Office
>> >>>>>>> Linnankatu 10 A
>> >>>>>>> FI-20100 Turku
>> >>>>>>> FINLAND
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>> --
>> www.lingsoft.fi
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>> Mikael Pesonen
>> System Engineer
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>> Time zone: GMT+2
>> Helsinki Office
>> Eteläranta 10
>> FI-00130 Helsinki
>> FINLAND
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
>>
>>
>>
>
> --
> www.lingsoft.fi
>
> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>
> Mikael Pesonen
> System Engineer
>
> e-mail: mikael.pesonen@lingsoft.fi
> Tel. +358 2 279 3300
>
> Time zone: GMT+2
>
> Helsinki Office
> Eteläranta 10
> FI-00130 Helsinki
> FINLAND
>
> Turku Office
> Linnankatu 10 A
> FI-20100 Turku
> FINLAND
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi Rob,
Im using this command to start db:
/usr/bin/java -Xmx3600M -jar
apache-jena-fuseki-2.3.1/fuseki-server.jar --update --port 3030
--loc=../apache-jena-3.0.1/DB /ds
and s- command line tools to make queries. In documentation there is
tdbquery which I'm not using. But TDB is still in use?
Thanks,
Mikael
On 26.8.2016 13:27, Rob Vesse wrote:
> Mikael
>
> If you\u2019re using Fuseki then you are using TDB already. TDB is a native RDF database that uses memory mapped files to make data access as fast as possible.
>
> SDB is a legacy system built on top of relational databases, so queries have to be compiled into SQL, submitted to the underlying relational database, and their results translated back into RDF appropriately. More complex queries cannot be translated directly into a single SQL query due to the differing semantics between the two query languages I may require many SQL Queries to answer. SDB is no longer actively developed and receives only minor bug fixes.
>
> As for when you would not use TDB no probably three main criteria:
>
> 1 - the amount of data you will store is that into the billions of triples. TDB Will scale pretty well into the millions of triples although this will depend on the complexity of the queries.
> 2 - when you need clustering for load balancing, failover etc. TDB is a single node system, while there are ways to do load balancing these typically rely on layering additional services on top of it
> 3 - when you need reasoning support. TDB does not natively support reasoning, you can use other Jena apis to add this but they Will substantially be great performance because they require all the data to be in heap memory. If your data is static then you can compute the inference closure once and persist that into the database but if you need dynamic inference or extremely large-scale inference then TDB Will not be suitable.
>
> There are plenty of commercial options that do address the above three criteria and people can probably provide recommendations if you think you need a commercial option.
>
> It is also worth noting that some queries are simply hard for any query engine to answer
>
> Rob
>
> On 26/08/2016 10:46, "Mikael Pesonen" <mi...@lingsoft.fi> wrote:
>
>
> Hi, still wondering what I should do to make the performance better.
>
> I read that TDB is faster. What is the reason not to use TDB? Cant find
> any comparison on SDB and TDB in that regard.
>
> Br,
> Mikael
>
>
> On 16.8.2016 13:13, Andy Seaborne wrote:
> > On 15/08/16 09:47, Mikael Pesonen wrote:
> >>
> >> Hi,
> >>
> >> what do you mean by masking? It should remove duplicates and it makes
> >> the query run in half time compared to without DISTINCT. Result count at
> >> least is the same.
> >>
> >> Mikael
> >
> > If DISTINCT cause a lot of results to be turned into a few, it is
> > hiding a lot of work by the query engine.
> >
> > If it's the inner DISTINCT that halves the execution time, then the
> > improvements (in dev builds) to property* may help you.
> >
> > If it's the outer one, it's a serialization issue (which I doubt at
> > this sacale).
> >
> > Andy
> >
> >>
> >>
> >> On 12.8.2016 13:53, Andy Seaborne wrote:
> >>> On 08/08/16 11:56, Mikael Pesonen wrote:
> >>>>
> >>>> Hi Andy,
> >>>>
> >>>> storage is started like this:
> >>>>
> >>>> /usr/bin/java -Xmx3600M -jar
> >>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
> >>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
> >>>>
> >>>> Ontology data is simple SKOS, and document data is also simple DC
> >>>> metadata triplets. Query returns ~15k triplets.
> >>>>
> >>>> I tested the SKOS part, and this executed in less than one second,
> >>>> returning ~50 items:
> >>>
> >>> How many without the two DISTINCT?
> >>>
> >>> I am wondering if the DISTINCT (the inner one) is masking a lot of
> >>> results.
> >>>
> >>>>
> >>>> SELECT DISTINCT *
> >>>> WHERE {
> >>>> GRAPH ?graph {
> >>>> SELECT DISTINCT ?child WHERE {
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> UNION
> >>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
> >>>>
> >>>>
> >>>> skos:narrower* ?child}
> >>>> }
> >>>> }
> >>>> }
> >>>>
> >>>> Br,
> >>>> Mikael
> >>>>
> >>>>
> >>>> On 8.8.2016 13:43, Andy Seaborne wrote:
> >>>>> There is a a certain amount of "it depend" here: what's the data
> >>>>> stored in? what shape is the data?( which Jena version?)
> >>>>>
> >>>>> In the next release, and available in development builds is:
> >>>>>
> >>>>> https://issues.apache.org/jira/browse/JENA-1195
> >>>>>
> >>>>> where property* got speeded up recently. Usually, it took moderately
> >>>>> unusual data to show this up but the repeated use of an expensive
> >>>>> operation in property* may be happening here too.
> >>>>>
> >>>>> Mikael - are you able to try out a SNAPSHOT build?
> >>>>>
> >>>>> Andy
> >>>>>
> >>>>>
> >>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
> >>>>>> Is this any better?
> >>>>>>
> >>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
> >>>>>>
> >>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
> >>>>>> ?s <http://purl.org/dc/terms/isPartOf>
> >>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
> >>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
> >>>>>> ?s ?p ?o
> >>>>>> }
> >>>>>>
> >>>>>> GRAPH <http://www.lingsoft.fi/> {
> >>>>>> SELECT DISTINCT ?child WHERE {
> >>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
> >>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
> >>>>>> skos:narrower* ?child} UNION
> >>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
> >>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
> >>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
> >>>>>> skos:narrower* ?child} UNION
> >>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
> >>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
> >>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
> >>>>>> skos:narrower* ?child} UNION
> >>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
> >>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
> >>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
> >>>>>> skos:narrower* ?child} UNION
> >>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
> >>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
> >>>>>> ?child}
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>> }
> >>>>>>
> >>>>>> Regards,
> >>>>>> H�vard M. Ottestad
> >>>>>>
> >>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
> >>>>>>> <mi...@lingsoft.fi> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I'm not if this is the correct forum to ask but hope you can help.
> >>>>>>> This query takes over 20 seconds with jena:
> >>>>>>>
> >>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
> >>>>>>> SELECT DISTINCT ?child WHERE {
> >>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
> >>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
> >>>>>>> skos:narrower* ?child} UNION
> >>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
> >>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
> >>>>>>> skos:narrower* ?child} UNION
> >>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
> >>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
> >>>>>>> skos:narrower* ?child} UNION
> >>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
> >>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
> >>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
> >>>>>>> skos:narrower* ?child} UNION
> >>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
> >>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
> >>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
> >>>>>>> <http://purl.org/dc/terms/isPartOf>
> >>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
> >>>>>>> graph query is for getting keywords from an ontology graph, second
> >>>>>>> is for querying documents having those keywords. Is there better
> >>>>>>> way/order to make this query? Thank you for the help, Mikael
> >>>>>>>
> >>>>>>> --
> >>>>>>> www.lingsoft.fi
> >>>>>>>
> >>>>>>> Speech Applications - Language Management - Translation - Reader's
> >>>>>>> and Writer's Tools - Text Tools - E-books and M-books
> >>>>>>>
> >>>>>>> Mikael Pesonen
> >>>>>>> System Engineer
> >>>>>>>
> >>>>>>> e-mail: mikael.pesonen@lingsoft.fi
> >>>>>>> Tel. +358 2 279 3300
> >>>>>>>
> >>>>>>> Time zone: GMT+2
> >>>>>>>
> >>>>>>> Helsinki Office
> >>>>>>> Etel�ranta 10
> >>>>>>> FI-00130 Helsinki
> >>>>>>> FINLAND
> >>>>>>>
> >>>>>>> Turku Office
> >>>>>>> Linnankatu 10 A
> >>>>>>> FI-20100 Turku
> >>>>>>> FINLAND
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
> --
> www.lingsoft.fi
>
> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>
> Mikael Pesonen
> System Engineer
>
> e-mail: mikael.pesonen@lingsoft.fi
> Tel. +358 2 279 3300
>
> Time zone: GMT+2
>
> Helsinki Office
> Etel�ranta 10
> FI-00130 Helsinki
> FINLAND
>
> Turku Office
> Linnankatu 10 A
> FI-20100 Turku
> FINLAND
>
>
>
>
>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Mikael
If you’re using Fuseki then you are using TDB already. TDB is a native RDF database that uses memory mapped files to make data access as fast as possible.
SDB is a legacy system built on top of relational databases, so queries have to be compiled into SQL, submitted to the underlying relational database, and their results translated back into RDF appropriately. More complex queries cannot be translated directly into a single SQL query due to the differing semantics between the two query languages I may require many SQL Queries to answer. SDB is no longer actively developed and receives only minor bug fixes.
As for when you would not use TDB no probably three main criteria:
1 - the amount of data you will store is that into the billions of triples. TDB Will scale pretty well into the millions of triples although this will depend on the complexity of the queries.
2 - when you need clustering for load balancing, failover etc. TDB is a single node system, while there are ways to do load balancing these typically rely on layering additional services on top of it
3 - when you need reasoning support. TDB does not natively support reasoning, you can use other Jena apis to add this but they Will substantially be great performance because they require all the data to be in heap memory. If your data is static then you can compute the inference closure once and persist that into the database but if you need dynamic inference or extremely large-scale inference then TDB Will not be suitable.
There are plenty of commercial options that do address the above three criteria and people can probably provide recommendations if you think you need a commercial option.
It is also worth noting that some queries are simply hard for any query engine to answer
Rob
On 26/08/2016 10:46, "Mikael Pesonen" <mi...@lingsoft.fi> wrote:
Hi, still wondering what I should do to make the performance better.
I read that TDB is faster. What is the reason not to use TDB? Cant find
any comparison on SDB and TDB in that regard.
Br,
Mikael
On 16.8.2016 13:13, Andy Seaborne wrote:
> On 15/08/16 09:47, Mikael Pesonen wrote:
>>
>> Hi,
>>
>> what do you mean by masking? It should remove duplicates and it makes
>> the query run in half time compared to without DISTINCT. Result count at
>> least is the same.
>>
>> Mikael
>
> If DISTINCT cause a lot of results to be turned into a few, it is
> hiding a lot of work by the query engine.
>
> If it's the inner DISTINCT that halves the execution time, then the
> improvements (in dev builds) to property* may help you.
>
> If it's the outer one, it's a serialization issue (which I doubt at
> this sacale).
>
> Andy
>
>>
>>
>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>
>>>> Hi Andy,
>>>>
>>>> storage is started like this:
>>>>
>>>> /usr/bin/java -Xmx3600M -jar
>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>
>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>> metadata triplets. Query returns ~15k triplets.
>>>>
>>>> I tested the SKOS part, and this executed in less than one second,
>>>> returning ~50 items:
>>>
>>> How many without the two DISTINCT?
>>>
>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>> results.
>>>
>>>>
>>>> SELECT DISTINCT *
>>>> WHERE {
>>>> GRAPH ?graph {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> }
>>>> }
>>>> }
>>>>
>>>> Br,
>>>> Mikael
>>>>
>>>>
>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>
>>>>> In the next release, and available in development builds is:
>>>>>
>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>
>>>>> where property* got speeded up recently. Usually, it took moderately
>>>>> unusual data to show this up but the repeated use of an expensive
>>>>> operation in property* may be happening here too.
>>>>>
>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On 08/08/16 11:37, Håvard Ottestad wrote:
>>>>>> Is this any better?
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>> ?s ?p ?o
>>>>>> }
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>> ?child}
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Regards,
>>>>>> Håvard M. Ottestad
>>>>>>
>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>
>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>>>> graph query is for getting keywords from an ontology graph, second
>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>
>>>>>>> --
>>>>>>> www.lingsoft.fi
>>>>>>>
>>>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>
>>>>>>> Mikael Pesonen
>>>>>>> System Engineer
>>>>>>>
>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>> Tel. +358 2 279 3300
>>>>>>>
>>>>>>> Time zone: GMT+2
>>>>>>>
>>>>>>> Helsinki Office
>>>>>>> Eteläranta 10
>>>>>>> FI-00130 Helsinki
>>>>>>> FINLAND
>>>>>>>
>>>>>>> Turku Office
>>>>>>> Linnankatu 10 A
>>>>>>> FI-20100 Turku
>>>>>>> FINLAND
>>>>>>>
>>>>>
>>>>
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi, still wondering what I should do to make the performance better.
I read that TDB is faster. What is the reason not to use TDB? Cant find
any comparison on SDB and TDB in that regard.
Br,
Mikael
On 16.8.2016 13:13, Andy Seaborne wrote:
> On 15/08/16 09:47, Mikael Pesonen wrote:
>>
>> Hi,
>>
>> what do you mean by masking? It should remove duplicates and it makes
>> the query run in half time compared to without DISTINCT. Result count at
>> least is the same.
>>
>> Mikael
>
> If DISTINCT cause a lot of results to be turned into a few, it is
> hiding a lot of work by the query engine.
>
> If it's the inner DISTINCT that halves the execution time, then the
> improvements (in dev builds) to property* may help you.
>
> If it's the outer one, it's a serialization issue (which I doubt at
> this sacale).
>
> Andy
>
>>
>>
>> On 12.8.2016 13:53, Andy Seaborne wrote:
>>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>>
>>>> Hi Andy,
>>>>
>>>> storage is started like this:
>>>>
>>>> /usr/bin/java -Xmx3600M -jar
>>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>>
>>>> Ontology data is simple SKOS, and document data is also simple DC
>>>> metadata triplets. Query returns ~15k triplets.
>>>>
>>>> I tested the SKOS part, and this executed in less than one second,
>>>> returning ~50 items:
>>>
>>> How many without the two DISTINCT?
>>>
>>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>>> results.
>>>
>>>>
>>>> SELECT DISTINCT *
>>>> WHERE {
>>>> GRAPH ?graph {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> UNION
>>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>>
>>>>
>>>> skos:narrower* ?child}
>>>> }
>>>> }
>>>> }
>>>>
>>>> Br,
>>>> Mikael
>>>>
>>>>
>>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>>> There is a a certain amount of "it depend" here: what's the data
>>>>> stored in? what shape is the data?( which Jena version?)
>>>>>
>>>>> In the next release, and available in development builds is:
>>>>>
>>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>>
>>>>> where property* got speeded up recently. Usually, it took moderately
>>>>> unusual data to show this up but the repeated use of an expensive
>>>>> operation in property* may be happening here too.
>>>>>
>>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>>>> Is this any better?
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>>> ?s ?p ?o
>>>>>> }
>>>>>>
>>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>>> ?child}
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Regards,
>>>>>> H�vard M. Ottestad
>>>>>>
>>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>>>> This query takes over 20 seconds with jena:
>>>>>>>
>>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>>> skos:narrower* ?child} UNION
>>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>>>> graph query is for getting keywords from an ontology graph, second
>>>>>>> is for querying documents having those keywords. Is there better
>>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>>
>>>>>>> --
>>>>>>> www.lingsoft.fi
>>>>>>>
>>>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>>
>>>>>>> Mikael Pesonen
>>>>>>> System Engineer
>>>>>>>
>>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>>> Tel. +358 2 279 3300
>>>>>>>
>>>>>>> Time zone: GMT+2
>>>>>>>
>>>>>>> Helsinki Office
>>>>>>> Etel�ranta 10
>>>>>>> FI-00130 Helsinki
>>>>>>> FINLAND
>>>>>>>
>>>>>>> Turku Office
>>>>>>> Linnankatu 10 A
>>>>>>> FI-20100 Turku
>>>>>>> FINLAND
>>>>>>>
>>>>>
>>>>
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 15/08/16 09:47, Mikael Pesonen wrote:
>
> Hi,
>
> what do you mean by masking? It should remove duplicates and it makes
> the query run in half time compared to without DISTINCT. Result count at
> least is the same.
>
> Mikael
If DISTINCT cause a lot of results to be turned into a few, it is hiding
a lot of work by the query engine.
If it's the inner DISTINCT that halves the execution time, then the
improvements (in dev builds) to property* may help you.
If it's the outer one, it's a serialization issue (which I doubt at this
sacale).
Andy
>
>
> On 12.8.2016 13:53, Andy Seaborne wrote:
>> On 08/08/16 11:56, Mikael Pesonen wrote:
>>>
>>> Hi Andy,
>>>
>>> storage is started like this:
>>>
>>> /usr/bin/java -Xmx3600M -jar
>>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>>
>>> Ontology data is simple SKOS, and document data is also simple DC
>>> metadata triplets. Query returns ~15k triplets.
>>>
>>> I tested the SKOS part, and this executed in less than one second,
>>> returning ~50 items:
>>
>> How many without the two DISTINCT?
>>
>> I am wondering if the DISTINCT (the inner one) is masking a lot of
>> results.
>>
>>>
>>> SELECT DISTINCT *
>>> WHERE {
>>> GRAPH ?graph {
>>> SELECT DISTINCT ?child WHERE {
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>>
>>> skos:narrower* ?child}
>>> UNION
>>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>>
>>> skos:narrower* ?child}
>>> }
>>> }
>>> }
>>>
>>> Br,
>>> Mikael
>>>
>>>
>>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>>> There is a a certain amount of "it depend" here: what's the data
>>>> stored in? what shape is the data?( which Jena version?)
>>>>
>>>> In the next release, and available in development builds is:
>>>>
>>>> https://issues.apache.org/jira/browse/JENA-1195
>>>>
>>>> where property* got speeded up recently. Usually, it took moderately
>>>> unusual data to show this up but the repeated use of an expensive
>>>> operation in property* may be happening here too.
>>>>
>>>> Mikael - are you able to try out a SNAPSHOT build?
>>>>
>>>> Andy
>>>>
>>>>
>>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>>> Is this any better?
>>>>>
>>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>>
>>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>>> ?s ?p ?o
>>>>> }
>>>>>
>>>>> GRAPH <http://www.lingsoft.fi/> {
>>>>> SELECT DISTINCT ?child WHERE {
>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>>> ?child}
>>>>> }
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> Regards,
>>>>> H�vard M. Ottestad
>>>>>
>>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>>> <mi...@lingsoft.fi> wrote:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>>> This query takes over 20 seconds with jena:
>>>>>>
>>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>>> SELECT DISTINCT ?child WHERE {
>>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>>> skos:narrower* ?child} UNION
>>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>>> graph query is for getting keywords from an ontology graph, second
>>>>>> is for querying documents having those keywords. Is there better
>>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>>
>>>>>> --
>>>>>> www.lingsoft.fi
>>>>>>
>>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>>
>>>>>> Mikael Pesonen
>>>>>> System Engineer
>>>>>>
>>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>>> Tel. +358 2 279 3300
>>>>>>
>>>>>> Time zone: GMT+2
>>>>>>
>>>>>> Helsinki Office
>>>>>> Etel�ranta 10
>>>>>> FI-00130 Helsinki
>>>>>> FINLAND
>>>>>>
>>>>>> Turku Office
>>>>>> Linnankatu 10 A
>>>>>> FI-20100 Turku
>>>>>> FINLAND
>>>>>>
>>>>
>>>
>>
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi,
what do you mean by masking? It should remove duplicates and it makes
the query run in half time compared to without DISTINCT. Result count at
least is the same.
Mikael
On 12.8.2016 13:53, Andy Seaborne wrote:
> On 08/08/16 11:56, Mikael Pesonen wrote:
>>
>> Hi Andy,
>>
>> storage is started like this:
>>
>> /usr/bin/java -Xmx3600M -jar
>> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
>> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>>
>> Ontology data is simple SKOS, and document data is also simple DC
>> metadata triplets. Query returns ~15k triplets.
>>
>> I tested the SKOS part, and this executed in less than one second,
>> returning ~50 items:
>
> How many without the two DISTINCT?
>
> I am wondering if the DISTINCT (the inner one) is masking a lot of
> results.
>
>>
>> SELECT DISTINCT *
>> WHERE {
>> GRAPH ?graph {
>> SELECT DISTINCT ?child WHERE {
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
>>
>> skos:narrower* ?child}
>> UNION
>> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
>>
>> skos:narrower* ?child}
>> }
>> }
>> }
>>
>> Br,
>> Mikael
>>
>>
>> On 8.8.2016 13:43, Andy Seaborne wrote:
>>> There is a a certain amount of "it depend" here: what's the data
>>> stored in? what shape is the data?( which Jena version?)
>>>
>>> In the next release, and available in development builds is:
>>>
>>> https://issues.apache.org/jira/browse/JENA-1195
>>>
>>> where property* got speeded up recently. Usually, it took moderately
>>> unusual data to show this up but the repeated use of an expensive
>>> operation in property* may be happening here too.
>>>
>>> Mikael - are you able to try out a SNAPSHOT build?
>>>
>>> Andy
>>>
>>>
>>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>>> Is this any better?
>>>>
>>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>>
>>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>>> ?s <http://purl.org/dc/terms/isPartOf>
>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>>> ?s ?p ?o
>>>> }
>>>>
>>>> GRAPH <http://www.lingsoft.fi/> {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower*
>>>> ?child}
>>>> }
>>>> }
>>>>
>>>> }
>>>>
>>>> Regards,
>>>> H�vard M. Ottestad
>>>>
>>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>>> <mi...@lingsoft.fi> wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>>> This query takes over 20 seconds with jena:
>>>>>
>>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>>> SELECT DISTINCT ?child WHERE {
>>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>>> skos:narrower* ?child} UNION
>>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>>> <http://purl.org/dc/terms/isPartOf>
>>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>>> graph query is for getting keywords from an ontology graph, second
>>>>> is for querying documents having those keywords. Is there better
>>>>> way/order to make this query? Thank you for the help, Mikael
>>>>>
>>>>> --
>>>>> www.lingsoft.fi
>>>>>
>>>>> Speech Applications - Language Management - Translation - Reader's
>>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>>
>>>>> Mikael Pesonen
>>>>> System Engineer
>>>>>
>>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>>> Tel. +358 2 279 3300
>>>>>
>>>>> Time zone: GMT+2
>>>>>
>>>>> Helsinki Office
>>>>> Etel�ranta 10
>>>>> FI-00130 Helsinki
>>>>> FINLAND
>>>>>
>>>>> Turku Office
>>>>> Linnankatu 10 A
>>>>> FI-20100 Turku
>>>>> FINLAND
>>>>>
>>>
>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
On 08/08/16 11:56, Mikael Pesonen wrote:
>
> Hi Andy,
>
> storage is started like this:
>
> /usr/bin/java -Xmx3600M -jar
> /home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
> --port 3030 --loc=../apache-jena-3.0.1/DB /ds
>
> Ontology data is simple SKOS, and document data is also simple DC
> metadata triplets. Query returns ~15k triplets.
>
> I tested the SKOS part, and this executed in less than one second,
> returning ~50 items:
How many without the two DISTINCT?
I am wondering if the DISTINCT (the inner one) is masking a lot of results.
>
> SELECT DISTINCT *
> WHERE {
> GRAPH ?graph {
> SELECT DISTINCT ?child WHERE {
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
> skos:narrower* ?child}
> UNION
> {<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
> skos:narrower* ?child}
> }
> }
> }
>
> Br,
> Mikael
>
>
> On 8.8.2016 13:43, Andy Seaborne wrote:
>> There is a a certain amount of "it depend" here: what's the data
>> stored in? what shape is the data?( which Jena version?)
>>
>> In the next release, and available in development builds is:
>>
>> https://issues.apache.org/jira/browse/JENA-1195
>>
>> where property* got speeded up recently. Usually, it took moderately
>> unusual data to show this up but the repeated use of an expensive
>> operation in property* may be happening here too.
>>
>> Mikael - are you able to try out a SNAPSHOT build?
>>
>> Andy
>>
>>
>> On 08/08/16 11:37, H�vard Ottestad wrote:
>>> Is this any better?
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE {
>>>
>>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>>> ?s <http://purl.org/dc/terms/isPartOf>
>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>>> ?s <http://purl.org/dc/terms/subject> ?child .
>>> ?s ?p ?o
>>> }
>>>
>>> GRAPH <http://www.lingsoft.fi/> {
>>> SELECT DISTINCT ?child WHERE {
>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>>> }
>>> }
>>>
>>> }
>>>
>>> Regards,
>>> H�vard M. Ottestad
>>>
>>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>>> <mi...@lingsoft.fi> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm not if this is the correct forum to ask but hope you can help.
>>>> This query takes over 20 seconds with jena:
>>>>
>>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>>> SELECT DISTINCT ?child WHERE {
>>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>>> skos:narrower* ?child} UNION
>>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>>> <http://purl.org/dc/terms/isPartOf>
>>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>>> graph query is for getting keywords from an ontology graph, second
>>>> is for querying documents having those keywords. Is there better
>>>> way/order to make this query? Thank you for the help, Mikael
>>>>
>>>> --
>>>> www.lingsoft.fi
>>>>
>>>> Speech Applications - Language Management - Translation - Reader's
>>>> and Writer's Tools - Text Tools - E-books and M-books
>>>>
>>>> Mikael Pesonen
>>>> System Engineer
>>>>
>>>> e-mail: mikael.pesonen@lingsoft.fi
>>>> Tel. +358 2 279 3300
>>>>
>>>> Time zone: GMT+2
>>>>
>>>> Helsinki Office
>>>> Etel�ranta 10
>>>> FI-00130 Helsinki
>>>> FINLAND
>>>>
>>>> Turku Office
>>>> Linnankatu 10 A
>>>> FI-20100 Turku
>>>> FINLAND
>>>>
>>
>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Hi Andy,
storage is started like this:
/usr/bin/java -Xmx3600M -jar
/home/text/tools/apache-jena-fuseki-2.3.1/fuseki-server.jar --update
--port 3030 --loc=../apache-jena-3.0.1/DB /ds
Ontology data is simple SKOS, and document data is also simple DC
metadata triplets. Query returns ~15k triplets.
I tested the SKOS part, and this executed in less than one second,
returning ~50 items:
SELECT DISTINCT *
WHERE {
GRAPH ?graph {
SELECT DISTINCT ?child WHERE {
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/c16e9937a515bda6>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/e56f6309f0d86b95>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b393055ac0f3a0bc>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/642194686a67f935>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/a9beeb4bf0b0af70>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/ce3598292f301cec>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/26aa300e4c033981>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bd07d765f36ea88f>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/bcf9e082e2ae8c9b>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/78d3955357a8ac10>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/369b1a9c822f55db>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/7098a84669b9feca>
skos:narrower* ?child}
UNION
{<http://www.lingsoft.fi/ontologies/VerohallintoAsiakaskirjeet/b7cb30c4efed996a>
skos:narrower* ?child}
}
}
}
Br,
Mikael
On 8.8.2016 13:43, Andy Seaborne wrote:
> There is a a certain amount of "it depend" here: what's the data
> stored in? what shape is the data?( which Jena version?)
>
> In the next release, and available in development builds is:
>
> https://issues.apache.org/jira/browse/JENA-1195
>
> where property* got speeded up recently. Usually, it took moderately
> unusual data to show this up but the repeated use of an expensive
> operation in property* may be happening here too.
>
> Mikael - are you able to try out a SNAPSHOT build?
>
> Andy
>
>
> On 08/08/16 11:37, H�vard Ottestad wrote:
>> Is this any better?
>>
>> SELECT DISTINCT ?s ?p ?o WHERE {
>>
>> GRAPH <http://www.lingsoft.fi/resource-meta/> {
>> ?s <http://purl.org/dc/terms/isPartOf>
>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
>> ?s <http://purl.org/dc/terms/subject> ?child .
>> ?s ?p ?o
>> }
>>
>> GRAPH <http://www.lingsoft.fi/> {
>> SELECT DISTINCT ?child WHERE {
>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower*
>> ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db>
>> skos:narrower* ?child} UNION
>> {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child}
>> UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
>> }
>> }
>>
>> }
>>
>> Regards,
>> H�vard M. Ottestad
>>
>>> On 08 Aug 2016, at 11:25, Mikael Pesonen
>>> <mi...@lingsoft.fi> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I'm not if this is the correct forum to ask but hope you can help.
>>> This query takes over 20 seconds with jena:
>>>
>>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> {
>>> SELECT DISTINCT ?child WHERE {
>>> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child}
>>> UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower*
>>> ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca>
>>> skos:narrower* ?child} UNION
>>> {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} }
>>> } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s
>>> <http://purl.org/dc/terms/subject> ?child . ?s
>>> <http://purl.org/dc/terms/isPartOf>
>>> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First
>>> graph query is for getting keywords from an ontology graph, second
>>> is for querying documents having those keywords. Is there better
>>> way/order to make this query? Thank you for the help, Mikael
>>>
>>> --
>>> www.lingsoft.fi
>>>
>>> Speech Applications - Language Management - Translation - Reader's
>>> and Writer's Tools - Text Tools - E-books and M-books
>>>
>>> Mikael Pesonen
>>> System Engineer
>>>
>>> e-mail: mikael.pesonen@lingsoft.fi
>>> Tel. +358 2 279 3300
>>>
>>> Time zone: GMT+2
>>>
>>> Helsinki Office
>>> Etel�ranta 10
>>> FI-00130 Helsinki
>>> FINLAND
>>>
>>> Turku Office
>>> Linnankatu 10 A
>>> FI-20100 Turku
>>> FINLAND
>>>
>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Etel�ranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Andy Seaborne <an...@apache.org>.
There is a a certain amount of "it depend" here: what's the data stored
in? what shape is the data?( which Jena version?)
In the next release, and available in development builds is:
https://issues.apache.org/jira/browse/JENA-1195
where property* got speeded up recently. Usually, it took moderately
unusual data to show this up but the repeated use of an expensive
operation in property* may be happening here too.
Mikael - are you able to try out a SNAPSHOT build?
Andy
On 08/08/16 11:37, Håvard Ottestad wrote:
> Is this any better?
>
> SELECT DISTINCT ?s ?p ?o WHERE {
>
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
> ?s <http://purl.org/dc/terms/subject> ?child .
> ?s ?p ?o
> }
>
> GRAPH <http://www.lingsoft.fi/> {
> SELECT DISTINCT ?child WHERE {
> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
> }
> }
>
> }
>
> Regards,
> Håvard M. Ottestad
>
>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>
>>
>> Hi,
>>
>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>
>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>
>> --
>> www.lingsoft.fi
>>
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>
>> Mikael Pesonen
>> System Engineer
>>
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>>
>> Time zone: GMT+2
>>
>> Helsinki Office
>> Eteläranta 10
>> FI-00130 Helsinki
>> FINLAND
>>
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
Re: Slow SPARQL query
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Thank you for the answer! That order seemed to be a bit slower.
Br,
Mikael
On 8.8.2016 13:37, Håvard Ottestad wrote:
> Is this any better?
>
> SELECT DISTINCT ?s ?p ?o WHERE {
>
> GRAPH <http://www.lingsoft.fi/resource-meta/> {
> ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
> ?s <http://purl.org/dc/terms/subject> ?child .
> ?s ?p ?o
> }
>
> GRAPH <http://www.lingsoft.fi/> {
> SELECT DISTINCT ?child WHERE {
> {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
> }
> }
>
> }
>
> Regards,
> Håvard M. Ottestad
>
>> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>>
>>
>> Hi,
>>
>> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>>
>> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>>
>> --
>> www.lingsoft.fi
>>
>> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>>
>> Mikael Pesonen
>> System Engineer
>>
>> e-mail: mikael.pesonen@lingsoft.fi
>> Tel. +358 2 279 3300
>>
>> Time zone: GMT+2
>>
>> Helsinki Office
>> Eteläranta 10
>> FI-00130 Helsinki
>> FINLAND
>>
>> Turku Office
>> Linnankatu 10 A
>> FI-20100 Turku
>> FINLAND
>>
--
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Linnankatu 10 A
FI-20100 Turku
FINLAND
Re: Slow SPARQL query
Posted by Håvard Ottestad <hm...@gmail.com>.
Is this any better?
SELECT DISTINCT ?s ?p ?o WHERE {
GRAPH <http://www.lingsoft.fi/resource-meta/> {
?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> .
?s <http://purl.org/dc/terms/subject> ?child .
?s ?p ?o
}
GRAPH <http://www.lingsoft.fi/> {
SELECT DISTINCT ?child WHERE {
{<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child}
}
}
}
Regards,
Håvard M. Ottestad
> On 08 Aug 2016, at 11:25, Mikael Pesonen <mi...@lingsoft.fi> wrote:
>
>
> Hi,
>
> I'm not if this is the correct forum to ask but hope you can help. This query takes over 20 seconds with jena:
>
> SELECT DISTINCT ?s ?p ?o WHERE { GRAPH <http://www.lingsoft.fi/> { SELECT DISTINCT ?child WHERE { {<http://www.lingsoft.fi/c16e9937a515bda6> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/e56f6309f0d86b95> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b393055ac0f3a0bc> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/642194686a67f935> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/a9beeb4bf0b0af70> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/ce3598292f301cec> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/26aa300e4c033981> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bd07d765f36ea88f> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/bcf9e082e2ae8c9b> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/78d3955357a8ac10> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/369b1a9c822f55db> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/7098a84669b9feca> skos:narrower* ?child} UNION {<http://www.lingsoft.fi/b7cb30c4efed996a> skos:narrower* ?child} } } GRAPH <http://www.lingsoft.fi/resource-meta/> { ?s <http://purl.org/dc/terms/subject> ?child . ?s <http://purl.org/dc/terms/isPartOf> <http://www.lingsoft.fi/rdf/uid/574ef1a40236a> . ?s ?p ?o } }First graph query is for getting keywords from an ontology graph, second is for querying documents having those keywords. Is there better way/order to make this query? Thank you for the help, Mikael
>
> --
> www.lingsoft.fi
>
> Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
>
> Mikael Pesonen
> System Engineer
>
> e-mail: mikael.pesonen@lingsoft.fi
> Tel. +358 2 279 3300
>
> Time zone: GMT+2
>
> Helsinki Office
> Eteläranta 10
> FI-00130 Helsinki
> FINLAND
>
> Turku Office
> Linnankatu 10 A
> FI-20100 Turku
> FINLAND
>