You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@jena.apache.org by GitBox <gi...@apache.org> on 2022/05/16 17:18:04 UTC
[GitHub] [jena] Aklakan opened a new pull request, #1315: GH-1314 Bulk retrieval with SERVICE clause
Aklakan opened a new pull request, #1315:
URL: https://github.com/apache/jena/pull/1315
This PR adds support for bulk retrieval with the SERVICE clause (related issue #1314 ).
Its currently a draft but it might a good enough for a discussion.
* It introduces a base class QueryIterRepeatApplyBulk of which QueryIterRepeatApply is a specialization. The only difference is that the `nextStage` function receives the input QueryIterator rather than only an individual binding.
* QueryIterServiceBulk is the new implementation that should eventually supersede QueryIterService.
* The implementation retrieves a batch of bindings from the input interator w.r.t. to the constants `ARQ.serviceBulkRequestMaxItemCount` and `ARQ.serviceBulkRequestMaxByteSize`.
* Consecutive bindings (after substitution) that refer to the same binding are subject to form a bulk request. Conversely, the implementation is not yet so smart to group non-consecutive service references as it makes resources management more complex.
* If a batch of bindings has size 1 then the implementation performs the usual binding substitution.
* Queries created from batches with more than 1 item have a VALUES block injected. This block uses only the join variables and a special `idx` variable. Values of this variable are reference to the index of the binding in the batch. This way, a remote binding's parent (=input) binding can be identified using `idx`. The generated Query sorts by `idx` which allows for streaming because the remote bindings arrive in the same order as the parent bindings in the batch.
* There is a "partition" machinery in place that deconstructs a batch response in a manner such that for every parent binding the corresponding stream of remote bindings can be obtained. I.e. it provides a view as if each parent binding in the batch was executed individually with the conventional approach. This e.g. allows for caching the results of the service clause execution for on the level of every parent binding. This is a bit of work in progress because I have a use case of connecting a caching system to it and I am looking into designing the API around it.
## Rewrite Strategies
The rewriting strategy depends on the content of the SERVICE clause:
The following is an example load a model with people from DBpedia.
```java
Model model;
try (QueryExecution qe = QueryExecutionHTTP.newBuilder()
.endpoint("https://dbpedia.org/sparql")
.query("CONSTRUCT WHERE { ?s a <http://dbpedia.org/ontology/Person> } LIMIT 100")
.build()) {
model = qe.execConstruct();
}
A rewrite strategy to inject the bulk of bindings is chosen based on the content of the SERVICE clause:
### Rewrite for Absent Slice
If there is no slice in the SERVICE clause's query, then a query is created that essentially has a join between a values block on the left hand size and the content of the service clause on the right hand side.
If the service query uses group by then it becomes a sub-query.
The "outer query" has a sort condtion on `idx` added (probably it could be omitted on most triple stores because results should be in the right order), and any further sort conditions are appended.
```java
try (QueryExecution qe = QueryExecutionFactory.create(
"SELECT * { ?s a <http://dbpedia.org/ontology/Person> SERVICE <https://dbpedia.org/sparql> { { SELECT ?s ?p { ?s ?p ?o } ORDER BY ?p } } }", model)) {
qe.getContext().set(ARQ.serviceBulkRequestMaxItemCount, 15);
qe.getContext().set(ARQ.serviceBulkRequestMaxByteSize, 1500);
ResultSetMgr.write(System.out, qe.execSelect(), ResultSetLang.RS_JSON);
}
```
```sparql
SELECT ?__idx__ ?p
WHERE
{ VALUES ( ?__idx__ ?s ) {
( 0 <http://dbpedia.org/resource/1999_NHK_Trophy> )
( 1 <http://dbpedia.org/resource/1909_World_Figure_Skating_Championships> )
( 2 <http://dbpedia.org/resource/1914_World_Figure_Skating_Championships> )
( 3 <http://dbpedia.org/resource/1999_Women's_Five_Nations_Championship> )
}
?s ?p ?o
}
ORDER BY ASC(?__idx__) ?p
```
### Rewrite for Present Slice
In this case, for every parent binding a union member is created based on conventional substitution. If the batch has size 1 then the `idx` var is omitted (and no union is created).
```java
try (QueryExecution qe = QueryExecutionFactory.create(
"SELECT * { ?s a <http://dbpedia.org/ontology/Person> SERVICE <https://dbpedia.org/sparql> { { SELECT * { ?s ?p ?o } LIMIT 3 OFFSET 5 } } }",
model)) {
ResultSetMgr.write(System.out, qe.execSelect(), ResultSetLang.RS_JSON);
}
```
Output:
```sparql
SELECT *
WHERE
{ { { SELECT *
WHERE
{ <http://dbpedia.org/resource/1999_NHK_Trophy>
?p ?o
}
OFFSET 5
LIMIT 3
}
BIND(0 AS ?__idx__)
}
UNION
{ { { SELECT *
WHERE
{ <http://dbpedia.org/resource/1909_World_Figure_Skating_Championships>
?p ?o
}
OFFSET 5
LIMIT 3
}
BIND(1 AS ?__idx__)
}
# ...
}}}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1153588568
Would now be a good time to split the PR into the changes for service executor extensibility and the separate extra module?
Then the service executor extensibility can go into the next release while the other work continues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r929917259
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
I am not totally sure what setup you prefer here - the poms of which modules should be adjusted and how?
The current functionality of the serviceenhancer extension works with fuseki without the need for a fuseki module - the java service loader of the serviceenhancer registers an assembler that can be used to construct a dataset that can be exposed via a fuseki service.
That means that just adding the serviceenhancer plugin as a dependency to fuseki directly would so far be sufficient to enable configuration via assembler.
There is another issue however which I am not sure how to tackle best: Execution of the `<java:org.apache.jena.sparql.service.enhancer.function.cacheInvalidate>` function should be restricted to only authorized users. Otherwise malicious agents could disrupt a service by constantly invalidation all caches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932396699
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> create a separate endpoint API that can be protected.
The following setup with now works with Fuseki. TestDatasetAssemblerServiceEnhancer now also includes tests whether t
```turtle
<#testService> rdf:type fuseki:Service;
fuseki:name "test";
fuseki:dataset <#testDs> .
<#testServiceAdmin> rdf:type fuseki:Service;
fuseki:name "testAdmin";
fuseki:dataset <#testDsAdmin> .
<#testDs> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> .
<#testDsAdmin> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> ; ja:enableMgmt true .
<#testDsCore> a ja:MemoryDataset .
```
If `ja:enableMgmt` is true, then the assembler for DatasetServiceEnhancer creates a DatasetGraphWrapper
with a copy of the base dataset's context with an additional symEnableMgmt attribute set to true.
The `afn:cacheRm` method checks for the presence of the flag and fails if it is absent.
What namespaces do you suggest? Right now I just added to the ja / afn /apf namespaces which avoids having to deal with more namespaces but maybe you'd prefer a plugin specific namespace such as `<http[s]://jena.apache.org/extra/service-enhancer/{function/pfunction/assembler}#>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1201446139
> For the namespace I would prefer `se: http://jena.apache.org/service-enhancer#` with a dash (`-`) but if you insist on underscore `_` it's also fine with me. I think there is no need to make things more complex by introducing further namespaces for functions / property functions / assembers.
`-` (dash) is fine.
> Should I move all IRIs introduced by the plugin to that namespace? - e.g. `se:DatasetServiceEnhancer` instead of `ja:DatasetServiceEnhancer`? This would make it more self-contained.
Yes, that would be best.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r936701767
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
The plugin now only introduces the namespace `http://jena.apache.org/service-enhancer#` which is used for both ARQ context symbols and assembler configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931586813
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> The issue here is that these should actually be privileged functions not everyone is permitted to run.
That is bundling everything into SPARQL queries. This is not SQL.
Privileged operations do not need to go through SPARQL query or be restricted to SPARQL syntax.
One step necessary step is ban <java:>.
```
PREFIX sepf: <java:org.apache.jena.sparql.service.enhancer.pfunction.>
PREFIX sefn: <java:org.apache.jena.sparql.service.enhancer.function.>
```
These need registered URIs.
e.g. create a separate endpoint API that can be protected.
> The same principle could be applied to e.g. list and cancel (long) running queries
Which is what the tasks endpoints can be extended for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931351632
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
The security / ACL is relevant for the aforementioned feature of listing the cache content using a property function and for explicitly invalidating cache entries using a function:
```sparql
SELECT sefn:cacheInvalidate(?id) {
?id sepf:cacheList (<http://dbpedia.org/sparql> ?queryStr ?binding)
FILTER(CONSTAINS(?queryStr, '?s'))
}
```
The same principle could be applied to e.g. list and cancel (long) running queries with SPARQL such as via a yasgui akin to postgresql's `SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE state = 'active' and pid <> pg_backend_pid();`.
The issue here is that these should actually be privileged functions not everyone is permitted to run.
And yes, if there were additional SPARQL extension functions that allowed for arbitrary reads and modification of the context then of course it could reveal sensitive information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932396699
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> create a separate endpoint API that can be protected.
The following setup now works with Fuseki. TestDatasetAssemblerServiceEnhancer now also includes tests for that setup with expected success / failure result.
```turtle
<#testService> rdf:type fuseki:Service; fuseki:name "test"; fuseki:dataset <#testDs> .
<#testServiceAdmin> rdf:type fuseki:Service; fuseki:name "testAdmin"; fuseki:dataset <#testDsAdmin> .
<#testDs> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> .
<#testDsAdmin> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> ; ja:enableMgmt true .
<#testDsCore> a ja:MemoryDataset .
```
If `ja:enableMgmt` is true, then the assembler for DatasetServiceEnhancer creates a DatasetGraphWrapper
with a copy of the base dataset's context with an additional symEnableMgmt attribute set to true.
The `afn:cacheRm` method checks for the presence of the flag and fails if it is absent.
(I renamed cacheList and cacheInvalidate to cacheLs and cacheRm as probably its easier to remember when one is already familiar with linux file system commands or docker - e.g. `docker image|volume ls`, `rm foo`, etc.)
What namespaces do you suggest? Right now I just added to the ja / afn /apf namespaces which avoids having to deal with more namespaces but maybe you'd prefer a plugin specific namespace such as `<http[s]://jena.apache.org/extra/service-enhancer/{function/pfunction/assembler}#>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939684734
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
I noticed those potential resource leaks warnings - and I wonder if there is any meaningful design pattern to make those go away:
Essentially these methods allocate resources which are used in the set up of the query iterators. Because of the streaming approach they can only be closed once the iterator is closed. Wrapping them with try-with-resources does not make sense there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1203977539
I'd say the first version is complete; besides the question about the best way to get this plugin into a fuski and any unresolved issue I might have overlooked.
Work for the next version (besides fixing any reported bugs) would be:
- Clean up the util package - some utils may make sense to add to other modules such as ARQ or maybe they even exist in places I haven't found yet.
- Find better names for some classes
- Add support for persistent caches storage. I already have a file-based implementation in my own library but it needs to be ported to the interfaces of the service enhancer plugin. Once Jena switches to non-shaded guava it would also be possible to factor the cache storage API out into it's own dependency as actually this system is jena-independent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1143707947
This is growing a lot of functionality in the core of ARQ. I am concerned that increasing specific code is creeping into the general engine.
Could it be a separate system that uses the service extensibility mechanism? This would also mean development is not on a Jena PR.
Could it do a simple case? Reading a few input iterator bindings (say up to 10) and issuing a compound SERVICE for 10 items. Repeat. This is a reduction of calls by x10.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r930054820
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
Hm, it's actually quite common to have access control on functions (or database objects in general).
E.g. postgres has `GRANT ALL ON PROCEDURE <procedure_name> TO <role>;` and Virtuoso has something similar such as `grant execute on DB.DBA.SPARQL_DELETE_DICT_CONTENT to "SPARQL";`
With the functions one can e.g. filter out which cache entries to invalidate:
```sparql
PREFIX sepf: <java:org.apache.jena.sparql.service.enhancer.pfunction.>
PREFIX sefn: <java:org.apache.jena.sparql.service.enhancer.function.>
SELECT sefn:cacheInvalidate(?id) {
?id sepf:cacheList (?serviceIri ?queryStr ?binding)
FILTER(CONSTAINS(?queryStr, '?s'))
}
```
Yes, functions can get moved around by the optimizer but that's more an argument that one has to be aware of pitfalls rather than that the whole approach is flawed. At least, other DBMS have support for ACLs on functions.
And ideally I'd prefer if this kind of ACL could be done on the dataset/context level in order to have it decoupled from the HTTP framework's security. But retrofitting ACLs on sparql functions is another can of worms - maybe eventually it needs to be opened.
If you are suggesting to have cache control outside of the realm of sparql function extensions it don't think it make things easier / streamlined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r927636105
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
Review Comment:
I added a cache implementation based on guava and removed the caffeine dependency. At least it succeeds on the test cases but maybe it introduced some issues we haven't spotted yet. I yet need to test it with our own project setup.
There is the AsyncClaimingCache interface and implementation that extends an undelying caching system with support for "claiming entries" and adding "eviction guards". Claimed entries are stored in a separate map as long as there are open claims. Eviction guards are simply java predicates that can 'catch' entries that are evicted by the underlying caching system and for which the predicate evaluates to true. Claims and guards both serve the purpose to temporarily bypass the underlying cache's eviction policy and both need to be closed eventually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931224743
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
**About security**: I was exactly thinking about having some factory method that creates appropriate graph views from a base dataset, a security model and a given user (actor). This is pretty much what is being done in fuseki with DataAccessCtl and SecurityContext. Well, there is not an explicit DatasetGraph view for a certain agent, but the SecurityContext corresponds to an agent's permissions and there is `SecurityContext.createQueryExecution(Query, DatasetGraph)` to set up evaluation of a query on a dataset w.r.t. a set of permissions.
For security over functions and property functions it seems that what would be needed is an extended or new SecurityContext with additional allow/deny lists and a permission check at an appropriate place (probably in ARQ) upon function evaluation.
> But that only goes so far.
Hm, is there much more complexity to it?
So I think for the first release of _this_ PR the security of the cache management is not that critical but its likely to be eventually needed in practice.
> This particular case could be done by context+function registry and/or datasets with different setups over the same data.
Yes, that was what I was aiming for.
**W.r.t to fuseki modules:** We are already making use of https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/apache-jena-fuseki/fuseki-server (rather than https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/jena-fuseki-server/fuseki-main) which loads all jars from the `./extra` folder. Not sure why there are two scripts with seemingly similar purpose - is the latter becoming the new start script?
The criticized shade profile in the serviceenhancer module is there to create the the self-contained jar that can be placed into the `extras` folder and thus on fuseki's classpath. So I think the main question here is which jena maven module should I use an example setup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931263447
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
+ <profiles>
+ <profile>
+ <id>bundle</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>3.2.4</version>
Review Comment:
This is marked "resolved" but I don't see a change.
3.2.4 is out of date - 3.3.0 is the latest - we'll just get a dependabot PR.
Is 3.2.4 necessary for some reason?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939684734
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
I noticed those potential resource leaks warnings - and I wonder if there is any meaningful design pattern to make those go away:
Essentially these methods allocate resources which are used in the set up of the query iterators. Because of the streaming approach they can only be closed once the iterator is closed. Wrapping them with try-with-resources does not make sense there.
I tried suppressing some of them but it's kind of a catch-22: The suppressions themselves can generate warning depending on the IDE compiler settings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r929958500
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
The difference between a FusekiModule and a JenaSystem is that FusekiModules are drop-ins. The base Fuseki server build does not have a dependency on it. (That said - I see why a JenaSystem should not work as a drop-in as well for now.)
Pragmatically, we can't keep loading more into Fuseki - some dependencies are huge (GraalVM, for example).
Drop-ins are added to the classpath by the server script. I've push up a _draft_ fuseki script:
https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/jena-fuseki-server/fuseki-main
which picks up `lib/*`.
Fuseki/webapp already has FUSEKI_CLASSES but that's a bit harder for non-Java people.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r929967015
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> `<java:org.apache.jena.sparql.service.enhancer.function.cacheInvalidate>` function should be restricted to only authorized users
This isn't a good use of functions! Functions get moved around by the optimizer.
The way to do it is to separate Fuseki server endpoint for cache control with its own security setup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r927105526
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
Review Comment:
Switching to plain guava cache should be doable. The main benefit of caffeine is the async API with support for futures and cancellation. The main issue might be differences in atomicity of the callbacks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1144683643
> Could it be a separate system that uses the service extensibility mechanism?
The changes to ARQ are not as severe as it may seem because of all the other changes in this PR
The changes are exactly there there to enable the service extensibility mechanism:
> I am concerned that increasing specific code is creeping into the general engine.
Yes that was my concern too so that's why I extended the service extension system in order to decouple it. I already hinted somewhere in my remarks that actually this PR is two PRs: (1) Enhancing the extension system and (2) providing the new functionality on top of it. Both parts are functional, and the current state of (1) is powerful enough to cater for (2).
Conversely, (2) is my test-bed for whether (1) is a sufficient foundation.
However, (2) needs more testing and I have a dependency on my own caching library in there which I want to disentangle so finishing (2) is still a bit of work.
To summarize the service extension system:
* The only change in OpExecutor is to now use QueryIteratorServiceBulk rather than QueryIteratorService. Before the extension system one had to override the execute(OpService) method - any existing overrides won't break with my change.
* QueryIterService continues to exist (though it should be deprecated in favor of the extension system) if someone used it with a custom OpExecutor before.
* I factored out the subclass QueryIterRepeatApplyBulk from QueryIteratorRepeatApply. Nothing that extended QueryIteratorRepeatApply should break. The only difference lies in the nextStage method that either accepts a QueryIterator or a single Binding, respectively. The bulk version delegates to the single binding one.
* QueryIteratorServiceBulk: This is the implementation which bridges to the extension by accessing the ServiceExecutorRegistryBulk from the execCxt. By default the bulk registry has an ServiceExecutorBulk that delegates to our old non-bulk ServiceExecutorRegistry (also accessed from the execCxt).
My extension of (1) bulk retrieval, (2) caching, (3) correlate, and (4) self reference (SERVICE <urn:self> to allow for correlating back ("fetch 5 employees of each department")) are already only plugins to the new extension system.
The only hard coded part right now is the initial registration in ServiceExecutorRegistryBulk / ServiceExecutorRegistry.
> Could it be a separate system that uses the service extensibility mechanism?
Yes, wanted to leave this option open for the caching system - not because it's not mature enough, but it currently has a dependency on my own library (which is on maven central though) but which in turn uses non-shaded guava.
> Could it do a simple case? Reading a few input iterator bindings (say up to 10) and issuing a compound SERVICE for 10 items. Repeat. This is a reduction of calls by x10.
Yes, this already works:
```sparql
SELECT * WHERE {
SERVICE <https://dbpedia.org/sparql> {
{ SELECT * {
?s a <http://dbpedia.org/ontology/MusicalArtist>
} ORDER BY DESC(?s) LIMIT 5 }
}
SERVICE <bulk+20:https://dbpedia.org/sparql> {
{ SELECT * {
?s ?p ?o
} LIMIT 1 }
}
}
```
The second service generates and sends a UNION query to remote endpoint:
```
SELECT *
WHERE
...
UNION
{ { { SELECT * { <http://dbpedia.org/resource/Živorad_Nikolić> ?p ?o BIND(2 AS ?__idx__) } LIMIT 1 } }
UNION
{ { { SELECT * { <http://dbpedia.org/resource/Živan_Cvitković> ?p ?o BIND(3 AS ?__idx__) } LIMIT 1 }
UNION
...
}
```
Some remarks: `bulk+20` gives some control of bulk size to the user but it is restricted by ARQ.serviceBulkMaxBindingCount (currently 100). The `+` symbol is used because RFC3986 only allows these symbols in an URI scheme:
```
scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
```
I need to give the query generation another pass in order to group bindings into VALUES blocks again where possible and do some more testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1144801436
Argh, my example is bad. What I meant is that by adding options as 'prefixes' they may in the process be parsed as scheme and this only allows for a very restricted set of characters. Just think of bulk+20 coming first. The service executor Plugin cuts those away - but my concern is that possibly a pedantic sparql query parser may complain about illegal uris. The remainder ending up as a path is imo acceptable. So it's really just about the sparql parser seeing these special uris.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1180212650
It's rebased now.
In the next few days I'll work on more clean up (especially of the test cases) and the documentation - then this first version should be pretty much complete. I think around at the end of this week I can remove the draft flag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939541835
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
These are IDE warnings (IntelliJ does, or should, have the same.)
They aren't more than general housekeeping, except for the `@return` which causes a "return" section with no content.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1144749824
FYI: `bulk-20` isn't in the scheme; that ended at the first `:`.
The RFC 3986 components are:
```
Scheme|correlate|
Authority|null|
Path|bulk+20:https://dbpedia.org/sparql|
Query|null|
Fragment|null|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1165657981
Please could you use the syntax `GH-nnnn:`?
Having things look the same would help me with dealing with the multiple open PRs and issues.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931351632
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
The security / ACL is relevant for the aforementioned feature of listing the cache content using a property function and for explicitly invalidating cache entries using a function:
```sparql
SELECT sefn:cacheInvalidate(?id) {
?id sepf:cacheList (<http://dbpedia.org/sparql> ?queryStr ?binding) # Select all cache entries for DBpedia
FILTER(CONSTAINS(?queryStr, '?s')) # Filter those that contain '?s'
}
```
The same principle could be applied to e.g. list and cancel (long) running queries with SPARQL such as via a yasgui akin to postgresql's `SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE state = 'active' and pid <> pg_backend_pid();`.
The issue here is that these should actually be privileged functions not everyone is permitted to run.
So I'd like to keep ACL on functions separate from this PR, but it might be worthwhile to consider it as a separate future one.
And yes, if there were additional SPARQL extension functions that allowed for arbitrary reads and modification of the context then of course it could reveal sensitive information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931351632
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
The security / ACL is relevant for the aforementioned feature of listing the cache content using a property function and for explicitly invalidating cache entries using a function:
```sparql
SELECT sefn:cacheInvalidate(?id) {
?id sepf:cacheList (<http://dbpedia.org/sparql> ?queryStr ?binding) # Select all cache entries for DBpedia
FILTER(CONSTAINS(?queryStr, '?s')) # Filter those that contain '?s'
}
```
The same principle could be applied to e.g. list and cancel (long) running queries with SPARQL such as via a yasgui akin to postgresql's `SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE state = 'active' and pid <> pg_backend_pid();`.
The issue here is that these should actually be privileged functions not everyone is permitted to run.
And yes, if there were additional SPARQL extension functions that allowed for arbitrary reads and modification of the context then of course it could reveal sensitive information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932396699
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> create a separate endpoint API that can be protected.
The following setup with now works with Fuseki. TestDatasetAssemblerServiceEnhancer now also includes tests whether t
```turtle
<#testService> rdf:type fuseki:Service;
fuseki:name "test";
fuseki:dataset <#testDs> .
<#testServiceAdmin> rdf:type fuseki:Service;
fuseki:name "testAdmin";
fuseki:dataset <#testDsAdmin> .
<#testDs> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> .
<#testDsAdmin> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> ; ja:enableMgmt true .
<#testDsCore> a ja:MemoryDataset .
```
If `ja:enableMgmt` is true, then the assembler for DatasetServiceEnhancer creates a DatasetGraphWrapper
with a copy of the base dataset's context with an additional symEnableMgmt attribute set to true.
The `afn:cacheRm` method checks for the presence of the flag and fails if it is absent.
(I renamed cacheList and cacheInvalidate to cacheLs and cacheRm as probably its easier to use when one is already familiar with linux file system commands or docker - e.g. `docker image|volume ls`, `rm foo`, etc.)
What namespaces do you suggest? Right now I just added to the ja / afn /apf namespaces which avoids having to deal with more namespaces but maybe you'd prefer a plugin specific namespace such as `<http[s]://jena.apache.org/extra/service-enhancer/{function/pfunction/assembler}#>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1201449466
It's nearly release time!
If you think this is, or will be in a mergeable state in the next few days, with comments addressed and you want the jena-extras module to go it, then great. It'll be "experimental" in the release notes anyway to give space for refinement.
If you prefer to wait a cycle (bugs, improvements pending,...) - that is fine - whatever works best for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r930054820
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
Hm, it's actually quite common to have access control on functions (or database objects in general).
E.g. postgres has `GRANT ALL ON PROCEDURE <procedure_name> TO <role>;` and Virtuoso has something similar such as `grant execute on DB.DBA.SPARQL_DELETE_DICT_CONTENT to "SPARQL";`
With the functions one can e.g. filter out which cache entries to invalidate:
```sparql
PREFIX sepf: <java:org.apache.jena.sparql.service.enhancer.pfunction.>
PREFIX sefn: <java:org.apache.jena.sparql.service.enhancer.function.>
SELECT sefn:cacheInvalidate(?id) {
?id sepf:cacheList (?serviceIri ?queryStr ?binding)
FILTER(CONSTAINS(?queryStr, '?s'))
}
```
Yes, functions can get moved around by the optimizer but that's more an argument that one has to be aware of pitfalls rather than that the whole approach is flawed. At least, other DBMS have support for ACLs on functions.
And ideally I'd prefer if this kind of ACL could be done on the dataset/context level in order to have it decoupled from the HTTP framework's security. But retrofitting ACLs on sparql functions is another can of worms - maybe eventually it needs to be opened.
If you are suggesting having cache control outside of the realm of sparql function extensions it don't think it make things easier / streamlined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939569153
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
In my Eclipse 2022-06 I had to manually enable the mentioned warnings - they now seem to be on ignore by default (at least I clicked on restore defaults and they remained on ignore) - I am pretty sure they were enabled in the past though.
Anyway, all the mentioned ones plus some additional ones should now be fixed and I squashed - can you confirm?
- [x] Check deployment to our test bed to make sure that my changes didn't break the fuseki setup (seems to work)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939684734
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
I noticed those potential resource leaks warnings - and I wonder if there is any meaningful design pattern to make those go away:
Essentially these methods allocate resources which are used in the set up of the query iterators. Because of the streaming approach they can only be closed once the iterator is closed. Wrapping them with try-with-resources does not make sense there.
I tried suppressing some of them but it's kind of a catch-22: The suppressions themselves generate warnings depending on the IDE compiler settings.
Anyway, awesome seeing this system reaching its first milestone!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939682136
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
There are a few "info" warning but nothing that can't be sorted up automatically which I will do.
They are warning that suppressed warnings (for "resource") are hidden by other settings which I guess is whether resource leaks for potential leaks is enabled as well as definite resource leaks. I have them off because where Jena predates try-resource, there significant numbers of warnings and need design changes to fix.
So good to merge!
Phew!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r927457555
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
Review Comment:
I can find a few Apache project using Caffine but they don't ship Apache-built binaries which is an additional level of problems with GPL2-CPE. GPL2-CPE and Java is not a simple matter. (What is "linking"?)
Having something we can't include in a Fuseki binary is a high barrier to acceptance.
Caffine uses modules so (I think) excluding a jar isn't going to work if jena is used with modulepath.
This needs investigation. https://issues.apache.org/jira/browse/LEGAL-336.
What features of Caffine are used by this work? Does it make a difference? Given there is a lot of off-machine work going on, is the difference in caching going to be visible? Can we put this behind an interface? (CacheFactory)
(It would be great to get rid of the shaded Guava. Guava is more stable these days.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932396699
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> create a separate endpoint API that can be protected.
The following setup with now works with Fuseki. TestDatasetAssemblerServiceEnhancer now also includes tests for that setup with expected success / failure result.
```turtle
<#testService> rdf:type fuseki:Service;
fuseki:name "test";
fuseki:dataset <#testDs> .
<#testServiceAdmin> rdf:type fuseki:Service;
fuseki:name "testAdmin";
fuseki:dataset <#testDsAdmin> .
<#testDs> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> .
<#testDsAdmin> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> ; ja:enableMgmt true .
<#testDsCore> a ja:MemoryDataset .
```
If `ja:enableMgmt` is true, then the assembler for DatasetServiceEnhancer creates a DatasetGraphWrapper
with a copy of the base dataset's context with an additional symEnableMgmt attribute set to true.
The `afn:cacheRm` method checks for the presence of the flag and fails if it is absent.
(I renamed cacheList and cacheInvalidate to cacheLs and cacheRm as probably its easier to remember when one is already familiar with linux file system commands or docker - e.g. `docker image|volume ls`, `rm foo`, etc.)
What namespaces do you suggest? Right now I just added to the ja / afn /apf namespaces which avoids having to deal with more namespaces but maybe you'd prefer a plugin specific namespace such as `<http[s]://jena.apache.org/extra/service-enhancer/{function/pfunction/assembler}#>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931246208
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> About security:
Why does the service execution need additional security? This is all becoming a separate processing system. Without careful whole system consideration, there will be leaks (e.g. the context) that reveal information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1146350174
If passing additional fixed arguments to SERVICE is a requirement, we can change the ARQ parser, e.g.
`SERVICE <http://my.endpoint/sparql> ARGS "cache" { ... }`
where `ARGS` is optional and takes a single string.
This avoids needing to modify the SERVICE URI, which might not be a constant - e.g. injected by `QueryExecBuilder.substitution`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r929917259
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
I am not totally sure what setup you prefer here - the poms of which modules should be adjusted and how?
The current functionality of the serviceenhancer extension works with fuseki without the need for a fuseki module - the java service loader of the serviceenhancer registers an assembler that can be used to construct a dataset that can be exposed via a fuseki service.
That means that just adding the serviceenhancer plugin as a dependency to fuseki directly would so far be sufficient to enable configuration via assembler.
There is another issue however which I am not sure how to tackle best: Execution of the `<java:org.apache.jena.sparql.service.enhancer.function.cacheInvalidate>` function should be restricted to only authorized users. Otherwise malicious agents could disrupt a service by constantly invalidating all caches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931355751
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> is the latter becoming the new start script?
Script `apache-jena-fuseki/fuseki-server` runs `jena-fuseki-fulljar` (not `jena-fuseki-server`) renamed into `apache-jena-fuseki` as `fuseki-server.jar`.
The goal is replace `jena-fuseki-fulljar`, a webapp system, in favour of Fuseki/Main together with modules for the UI and admin functionality.
`jena-fuseki-war` remains but it will be the only webapp based form.
The webapp version does not support FusekiModules. Only Fuseki/Main has that extension mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932467063
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
Same namespace prefix as the assembler (see another comment). From a whole system POV, whether you want to split function/property function/assembler vocab or use the same namespace is a local style choice. Either works.
`/extra/` isn't necessary. (would we have "/service-enhancer" as well? No.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r936702973
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
Review Comment:
Pom is now cleaned up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r927045654
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
Review Comment:
What's wrong with Guava cache available through CacheFactory?
(all caching solution say "high-performance" as if there would be any that aimed for low performance!)
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
Review Comment:
Remove
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
Review Comment:
Version management and dependencyManagement in the root POM please.
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
+ <profiles>
+ <profile>
+ <id>bundle</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>3.2.4</version>
Review Comment:
The version is in the root POM.
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
There are better ways to do that in Fuseki/Main.
It's actually easier to separate building the extensions and packaging as a FusekiModule into two maven modules. Exclusion management then by <excludes> on artifacts.
It's certainly easier to test the FusekiModule.
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
+ <profiles>
+ <profile>
+ <id>bundle</id>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>3.2.4</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <artifactSet>
+ <excludes>
+ <!-- <exclude>org.apache.jena:*</exclude> -->
+ <exclude>classworlds:classworlds</exclude>
Review Comment:
I don't see
classworlds
jmock
*:xml-apis
junit (in runtime scope)
org.apache.maven:lib:tests
log4j:log4j (which is Log4J1)
in the dependencies.
Needs to exclude more if I understand the intention here:: specifically org.apache.jena:apache-jena-libs:pom and all dependencies.
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/algebra/TransformSE_JoinStrategy.java:
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.algebra;
+
+import java.util.HashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.jena.graph.Node;
+import org.apache.jena.sparql.algebra.Op;
+import org.apache.jena.sparql.algebra.OpVars;
+import org.apache.jena.sparql.algebra.TransformCopy;
+import org.apache.jena.sparql.algebra.op.OpConditional;
+import org.apache.jena.sparql.algebra.op.OpDisjunction;
+import org.apache.jena.sparql.algebra.op.OpFilter;
+import org.apache.jena.sparql.algebra.op.OpJoin;
+import org.apache.jena.sparql.algebra.op.OpLeftJoin;
+import org.apache.jena.sparql.algebra.op.OpSequence;
+import org.apache.jena.sparql.algebra.op.OpService;
+import org.apache.jena.sparql.core.Var;
+import org.apache.jena.sparql.engine.Rename;
+import org.apache.jena.sparql.expr.ExprList;
+import org.apache.jena.sparql.graph.NodeTransform;
+import org.apache.jena.sparql.graph.NodeTransformLib;
+import org.apache.jena.sparql.service.enhancer.impl.ServiceOpts;
+
+/**
+ * Checks for the presence of SERVICE <loop:> { }
+ * transforms those into linear joins using {@link OpSequence} / {@link OpDisjunction}
+ * and adjust variable scopes.
+ *
+ * All variables mentioned on the rhs which have the same reverse-renaming as variables
+ * visible on the lhs will be substituted with the lhs variant.
+ *
+ * TODO Variable scope is applied after this transform - so the renaming
Review Comment:
TODO isn't helpful in the javadoc.
##########
jena-extras/jena-serviceenhancer/src/test/java/org/apache/jena/sparql/service/enhancer/impl/TS_ResultSetLimits.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl;
+
+import java.util.IdentityHashMap;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+import org.apache.jena.query.Dataset;
+import org.apache.jena.query.DatasetFactory;
+import org.apache.jena.query.Query;
+import org.apache.jena.query.QueryExecution;
+import org.apache.jena.query.QueryFactory;
+import org.apache.jena.query.ResultSetFactory;
+import org.apache.jena.query.ResultSetFormatter;
+import org.apache.jena.query.ResultSetRewindable;
+import org.apache.jena.rdf.model.Model;
+import org.apache.jena.rdf.model.ModelFactory;
+import org.apache.jena.rdf.model.Property;
+import org.apache.jena.rdf.model.Resource;
+import org.apache.jena.rdf.model.ResourceFactory;
+import org.apache.jena.sparql.engine.iterator.QueryIterSlice;
+import org.apache.jena.sparql.service.ServiceExecutorRegistry;
+import org.apache.jena.sparql.service.enhancer.init.InitServiceEnhancer;
+import org.apache.jena.sparql.util.Context;
+import org.apache.jena.vocabulary.RDF;
+import org.apache.jena.vocabulary.RDFS;
+import org.junit.Assert;
+import org.junit.Test;
+
+public abstract class TS_ResultSetLimits {
+
+ protected String mode;
+
+ public TS_ResultSetLimits(String mode) {
+ this.mode = mode;
+ }
+
+ private static final Property hasEmployee = ResourceFactory.createProperty("urn:hasEmployee");
+ private static final Property Person = ResourceFactory.createProperty("urn:Person");
+ private static final Property Department = ResourceFactory.createProperty("urn:Department");
+
+ public static Model createModel(int departments) {
+ Model result = ModelFactory.createDefaultModel();
+ // int departments = 4;
+
+ for (int d = 1; d <= departments; ++d) {
+ Resource department = result.createResource("urn:dept" + d)
+ .addProperty(RDF.type, Department)
+ .addLiteral(RDFS.label, "Department " + d);
+
+
+ for (int e = 1; e <= (departments - d + 1); ++e) {
+ Resource person = result.createResource("urn:person" + e)
+ .addProperty(RDF.type, Person)
+ .addLiteral(RDFS.label, "Person " + e);
+
+ department.addProperty(hasEmployee, person);
+ }
+ }
+
+ return result;
+ }
+
+ /** Departments in ascending order */
+ @Test
+ // @Ignore
+ public void testLoop01_asc_limit1() {
+ Model model = createModel(4);
+ int rows = test(model, "SELECT * { { SELECT ?d { ?d a <urn:Department> } ORDER BY ASC(?d) } SERVICE <${mode}> { ?d <urn:hasEmployee> ?p }}", 1);
+ Assert.assertEquals(4, rows);
+ }
+
+ @Test
+ // @Ignore
+ public void testLoop01_asc_limit2() {
+ // System.err.println("testLoop01_asc_limit2");
+
+ Model model = createModel(4);
+ int rows = test(model, "SELECT * { { SELECT ?d { ?d a <urn:Department> } ORDER BY ASC(?d) } SERVICE <${mode}> { ?d <urn:hasEmployee> ?p }}", 2);
+ Assert.assertEquals(7, rows);
+ }
+
+ /** Departments in descending order */
+ @Test
+ // @Ignore
+ public void testLoop01_desc_limit1() {
+ Model model = createModel(4);
+ int rows = test(model, "SELECT * { { SELECT ?d { ?d a <urn:Department> } ORDER BY DESC(?d) } SERVICE <${mode}> { ?d <urn:hasEmployee> ?p }}", 1);
+ //int rows = test(model, "SELECT * { SELECT ?d { ?d a <urn:Department> } ORDER BY DESC(?d) }", 1);
+ Assert.assertEquals(4, rows);
+ }
+
+ @Test
+ // @Ignore
+ public void testLoop01_desc_limit2() {
+ // System.out.println("testLoop01_desc_limit2");
+
+ Model model = createModel(4);
+ int rows = test(model, "SELECT * { { SELECT ?d { ?d a <urn:Department> } ORDER BY DESC(?d) } SERVICE <${mode}> { ?d <urn:hasEmployee> ?p }}", 2);
+ Assert.assertEquals(7, rows);
+ }
+
+
+ /** There are exactly 10 results with a result set limit of 10 -
+ * so a separate request that only yields the end marker may be needed */
+ @Test
+ // @Ignore
+ public void testLoop01_asc_limit10() {
+ // System.err.println("testLoop01_asc_limit10");
+
+ Model model = createModel(4);
+ int rows = test(model, "SELECT * { { SELECT ?d { ?d a <urn:Department> } ORDER BY ASC(?d) } SERVICE <${mode}> { ?d <urn:hasEmployee> ?p }}", 10);
+ Assert.assertEquals(10, rows);
+ }
+
+
+ public int test(Model model, String rawQueryStr, int hiddenLimit) {
+ ServiceResultSizeCache.get().invalidateAll();
+ ServiceResponseCache.get().invalidateAll();
+ String queryStr = rawQueryStr.replaceAll(Pattern.quote("${mode}"), mode);
+ int result = testCore(model, queryStr, hiddenLimit);
+ return result;
+ }
+
+ private static final Map<Model, Dataset> modelToDataset = new IdentityHashMap<>();
+
+ public static Dataset identityWrap(Model model) {
+ Dataset result = modelToDataset.computeIfAbsent(model, DatasetFactory::wrap);
+ return result;
+ }
+
+ public static int testWithCleanCaches(Model model, String queryStr, int hiddenLimit) {
+ return testWithCleanCaches(identityWrap(model), queryStr, hiddenLimit);
+ }
+
+ public static int testWithCleanCaches(Dataset dataset, String queryStr, int hiddenLimit) {
+ ServiceResultSizeCache.get().invalidateAll();
+ ServiceResponseCache.get().invalidateAll();
+
+ int result = testCore(dataset, queryStr, hiddenLimit);
+ return result;
+ }
+
+
+ public static int testCore(Model model, String queryStr, int hiddenLimit) {
+ return testCore(identityWrap(model), queryStr, hiddenLimit);
+ }
+
+ public static int testCore(Dataset dataset, String queryStr, int hiddenLimit) {
+
+
+ Query query = QueryFactory.create(queryStr);
+
+ // Register a service plugin that slices the result
+ ServiceExecutorRegistry reg = ServiceExecutorRegistry.get().copy();
+/*
Review Comment:
Remove or explain with a comment.
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/impl/util/QueryIterSlottedBase.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl.util;
+
+import org.apache.jena.atlas.io.IndentedWriter;
+import org.apache.jena.atlas.io.PrintUtils;
+import org.apache.jena.atlas.iterator.IteratorSlotted;
+import org.apache.jena.atlas.lib.Lib;
+import org.apache.jena.shared.PrefixMapping;
+import org.apache.jena.sparql.engine.Plan;
+import org.apache.jena.sparql.engine.QueryIterator;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.serializer.SerializationContext;
+import org.apache.jena.sparql.util.QueryOutputUtils;
+
+/**
+ * QueryIterator implementation based on IteratorSlotted.
+ * Its purpose is to ease wrapping a non-QueryIterator as one based
+ * on a {@link #moveToNext()} method analogous to guava's AbstractIterator.
+ */
+public abstract class QueryIterSlottedBase<T>
Review Comment:
If you implement QueryIterator, then iterators won't be checked and unclosed ones
"QueryIter" as a name is already in use.
ARQ iterators use as a basename to indicate they are derived from QueryIter.
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/impl/util/BindingUtils.java:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl.util;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.LinkedHashSet;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+
+import org.apache.jena.graph.Node;
+import org.apache.jena.sparql.core.Var;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.binding.BindingBuilder;
+import org.apache.jena.sparql.expr.ExprEvalException;
+import org.apache.jena.sparql.graph.NodeTransformLib;
+import org.apache.jena.sparql.syntax.syntaxtransform.NodeTransformSubst;
+
+public class BindingUtils {
+ public static Binding project(Binding binding, Iterable<Var> vars) {
+ return project(binding, vars.iterator(), Collections.emptySet());
+ }
+
+ public static Binding project(Binding binding, Iterator<Var> vars, Set<Var> blacklist) {
+ BindingBuilder builder = BindingBuilder.create();
+
+ while (vars.hasNext()) {
+ Var var = vars.next();
+ if (!blacklist.contains(var)) {
Review Comment:
Bad choice of name. Please rename all mentions (VarUtils, BindingUtils).
Try `exclusions` or some such.
And why no pass just `vars` and
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/claimingcache/RefImpl.java:
##########
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.claimingcache;
+
+import java.util.Map;
+import java.util.WeakHashMap;
+import java.util.function.Consumer;
+
+import org.apache.jena.sparql.service.enhancer.impl.util.StackTraceUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Implementation of a {@link Ref}
+ *
+ * TODO Avoid needless synchronization; a ConcurrentHashMap may help;
Review Comment:
TODO's in javadoc don't seem right.
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
Please clean the code.
I get 52 warnings in Eclipse.
Missing overrides, unused imports, bad javadoc (various kinds), bad try-with-resources, unnecessary casts.
mvn (with java11) javadoc:javadoc has 86 lines of javadoc warnings
There are many commented code fragments.
There multiple adjacent blank lines.
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/impl/util/QueryIterSlottedBase.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl.util;
+
+import org.apache.jena.atlas.io.IndentedWriter;
+import org.apache.jena.atlas.io.PrintUtils;
+import org.apache.jena.atlas.iterator.IteratorSlotted;
+import org.apache.jena.atlas.lib.Lib;
+import org.apache.jena.shared.PrefixMapping;
+import org.apache.jena.sparql.engine.Plan;
+import org.apache.jena.sparql.engine.QueryIterator;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.serializer.SerializationContext;
+import org.apache.jena.sparql.util.QueryOutputUtils;
+
+/**
+ * QueryIterator implementation based on IteratorSlotted.
+ * Its purpose is to ease wrapping a non-QueryIterator as one based
+ * on a {@link #moveToNext()} method analogous to guava's AbstractIterator.
+ */
+public abstract class QueryIterSlottedBase<T>
Review Comment:
There isn't any use of ExecutionContext. Is there a reason for this?
This is used for tracking iterator and also how the Context is accessed. Even if not needed now, it may be in the future and retrofitting becomes a painful task.
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/impl/util/BindingUtils.java:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl.util;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.LinkedHashSet;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+
+import org.apache.jena.graph.Node;
+import org.apache.jena.sparql.core.Var;
+import org.apache.jena.sparql.engine.binding.Binding;
+import org.apache.jena.sparql.engine.binding.BindingBuilder;
+import org.apache.jena.sparql.expr.ExprEvalException;
+import org.apache.jena.sparql.graph.NodeTransformLib;
+import org.apache.jena.sparql.syntax.syntaxtransform.NodeTransformSubst;
+
+public class BindingUtils {
+ public static Binding project(Binding binding, Iterable<Var> vars) {
+ return project(binding, vars.iterator(), Collections.emptySet());
+ }
+
+ public static Binding project(Binding binding, Iterator<Var> vars, Set<Var> blacklist) {
+ BindingBuilder builder = BindingBuilder.create();
+
+ while (vars.hasNext()) {
+ Var var = vars.next();
+ if (!blacklist.contains(var)) {
Review Comment:
You could separate filtering and building by using `Iter.notFilter`.
##########
jena-extras/jena-serviceenhancer/src/test/java/org/apache/jena/sparql/service/enhancer/impl/TS_ResultSetLimits.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl;
+
+import java.util.IdentityHashMap;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+import org.apache.jena.query.Dataset;
+import org.apache.jena.query.DatasetFactory;
+import org.apache.jena.query.Query;
+import org.apache.jena.query.QueryExecution;
+import org.apache.jena.query.QueryFactory;
+import org.apache.jena.query.ResultSetFactory;
+import org.apache.jena.query.ResultSetFormatter;
+import org.apache.jena.query.ResultSetRewindable;
+import org.apache.jena.rdf.model.Model;
+import org.apache.jena.rdf.model.ModelFactory;
+import org.apache.jena.rdf.model.Property;
+import org.apache.jena.rdf.model.Resource;
+import org.apache.jena.rdf.model.ResourceFactory;
+import org.apache.jena.sparql.engine.iterator.QueryIterSlice;
+import org.apache.jena.sparql.service.ServiceExecutorRegistry;
+import org.apache.jena.sparql.service.enhancer.init.InitServiceEnhancer;
+import org.apache.jena.sparql.util.Context;
+import org.apache.jena.vocabulary.RDF;
+import org.apache.jena.vocabulary.RDFS;
+import org.junit.Assert;
+import org.junit.Test;
+
+public abstract class TS_ResultSetLimits {
+
+ protected String mode;
+
+ public TS_ResultSetLimits(String mode) {
+ this.mode = mode;
+ }
+
+ private static final Property hasEmployee = ResourceFactory.createProperty("urn:hasEmployee");
+ private static final Property Person = ResourceFactory.createProperty("urn:Person");
+ private static final Property Department = ResourceFactory.createProperty("urn:Department");
+
+ public static Model createModel(int departments) {
+ Model result = ModelFactory.createDefaultModel();
+ // int departments = 4;
+
+ for (int d = 1; d <= departments; ++d) {
+ Resource department = result.createResource("urn:dept" + d)
+ .addProperty(RDF.type, Department)
+ .addLiteral(RDFS.label, "Department " + d);
+
+
+ for (int e = 1; e <= (departments - d + 1); ++e) {
+ Resource person = result.createResource("urn:person" + e)
+ .addProperty(RDF.type, Person)
+ .addLiteral(RDFS.label, "Person " + e);
+
+ department.addProperty(hasEmployee, person);
+ }
+ }
+
+ return result;
+ }
+
+ /** Departments in ascending order */
+ @Test
+ // @Ignore
Review Comment:
Remove (many).
##########
jena-extras/jena-serviceenhancer/src/test/java/org/apache/jena/sparql/service/enhancer/impl/TS_ResultSetLimits.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl;
+
+import java.util.IdentityHashMap;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+import org.apache.jena.query.Dataset;
+import org.apache.jena.query.DatasetFactory;
+import org.apache.jena.query.Query;
+import org.apache.jena.query.QueryExecution;
+import org.apache.jena.query.QueryFactory;
+import org.apache.jena.query.ResultSetFactory;
+import org.apache.jena.query.ResultSetFormatter;
+import org.apache.jena.query.ResultSetRewindable;
+import org.apache.jena.rdf.model.Model;
+import org.apache.jena.rdf.model.ModelFactory;
+import org.apache.jena.rdf.model.Property;
+import org.apache.jena.rdf.model.Resource;
+import org.apache.jena.rdf.model.ResourceFactory;
+import org.apache.jena.sparql.engine.iterator.QueryIterSlice;
+import org.apache.jena.sparql.service.ServiceExecutorRegistry;
+import org.apache.jena.sparql.service.enhancer.init.InitServiceEnhancer;
+import org.apache.jena.sparql.util.Context;
+import org.apache.jena.vocabulary.RDF;
+import org.apache.jena.vocabulary.RDFS;
+import org.junit.Assert;
+import org.junit.Test;
+
+public abstract class TS_ResultSetLimits {
Review Comment:
In Jena "TS_" are test suites.
`AbstractTestResultSetLimits` would be the pattern.
Personal opion but names that might overlap or which don't suggest where they are can get confusing at scale. Including some context like "Service" might be good long term.
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
Review Comment:
What is using JCL?
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/impl/util/VarUtils.java:
##########
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl.util;
+
+import java.util.Collection;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.jena.ext.com.google.common.collect.BiMap;
+import org.apache.jena.ext.com.google.common.collect.HashBiMap;
+import org.apache.jena.sparql.ARQConstants;
+import org.apache.jena.sparql.core.Var;
+import org.apache.jena.sparql.engine.Rename;
+
+public class VarUtils {
Review Comment:
Name already in-use.
##########
jena-extras/jena-serviceenhancer/src/main/java/org/apache/jena/sparql/service/enhancer/claimingcache/RefImpl.java:
##########
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.claimingcache;
+
+import java.util.Map;
+import java.util.WeakHashMap;
+import java.util.function.Consumer;
+
+import org.apache.jena.sparql.service.enhancer.impl.util.StackTraceUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Implementation of a {@link Ref}
+ *
+ * TODO Avoid needless synchronization; a ConcurrentHashMap may help;
+ * ref.equals could do only reference comparison
+ *
+ * @author raven
Review Comment:
Remove all `@author`.
##########
jena-extras/jena-serviceenhancer/README.md:
##########
@@ -0,0 +1,259 @@
+# Service Enhancer Plugin
+
+This plugin extends the functionality of the SERVICE clause with:
+
+- Bulk requests
+- Correlated joins also known as lateral joins
+- Caching
+
+As a fundamental principle, a request making use of cache and bulk should return the exact same result as if
+those settings were omitted. As a consequence runtime result set size recognization (RRR) is employed to reveal hidden
+result set limits and ensure that always only the appropriate amount of data is yeld from the caches.
Review Comment:
"yeld" -> "returned"
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
Review Comment:
It has a GPL2-with-classpath-exception dependency (org.checkerframework:checker-qual).
This is not acceptable for ASF projects.
It is not the same situation as the JDK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1191922804
For the release, I suggest a separate email message and sent by you (@Aklakan) that introduces this new feature in a way that does not have the distractions of the general coverage of [ANN]. This would be sent soon after after the AN Nand maybe also to a wider set of lists.
And jena-site material. The link can be tweeted.
README.md in the code base isn't the easiest or natural place for users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1146196013
Separating it out into two PRs is a good idea.
The only part to go into jena-arq is the extension mechanism changes such as wikidata might use.
The extension itself can be a separate maven module. We have the `jena-extras` area for things like this. Items in this area can evolve more easily than the core Jena system (roughly Fuskei+text) and be treated as evolving and labelled "experimental". The project does have the convention that material not in the core system does not block a release. A module can be skipped - not ideal but a stable core system has to be main focus.
There are lots of issues for the bulk extensions that aren't clear - for example the interactions a query that combines local patterns with SERVICE: "SERVICE - local - SERVICE". (And there is quite a lot of papers on the subject, some going back a few decades - research was ahead of the capabilities of networking and hardware!)
I want to enable easy evolution of the bulk-caching extension. Putting it in an optional module means that you have control over its evolution and it can mature there. Adding to `jena-arq` requires stability, long-term commitment to the exact behaviour and understanding the support+maintenance over the coming 5+ years. It also requires a lot more technical review and finding the time to do that each PR is going to slow things down. Hopefully a sub-community will
> QueryIterRepeatApplyBulk from QueryIteratorRepeatApply. Nothing that extended QueryIteratorRepeatApply should break.
QueryIteratorRepeatApply is central throughout execution. I don't see any changes to `QueryIterRepeatApplyBulk`.
> QueryIteratorServiceBulk rather than QueryIteratorService
Surely that's just `QueryIteratorService` - only the extension mechanism.
The extension itself executed there can be no more than a call through to the the functionality.
```
public interface ServiceExecution {
public QueryIterator exec();
}
```
that could have a `start` (or `initExecution`) and `finish` to get a full lifecycle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1166306627
Do you mean on the commit messages? I thought its only/mostly relevant upon squashing after successful review but sure I can do it during development too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931351632
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
The security / ACL is relevant for the aforementioned feature of listing the cache content using a property function and for explicitly invalidating cache entries using a function:
```sparql
SELECT sefn:cacheInvalidate(?id) {
?id sepf:cacheList (<http://dbpedia.org/sparql> ?queryStr ?binding) # Select all cache entries for DBpedia
FILTER(CONSTAINS(?queryStr, '?s')) # Filter those that contain '?s'
}
```
The same principle could be applied to e.g. list and cancel (long) running queries with SPARQL via a yasgui akin to postgresql's `SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE state = 'active' and pid <> pg_backend_pid();`.
The issue here is that these should actually be privileged functions not everyone is permitted to run.
So I'd like to keep ACL on functions separate from this PR, but it might be worthwhile to consider it as a separate future one.
And yes, if there were additional SPARQL extension functions that allowed for arbitrary reads and modification of the context then of course it could reveal sensitive information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931224743
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
**About security**: I was exactly thinking about having some factory method that creates appropriate graph views from a base dataset, a security model and a given user (actor). This is pretty much what is being done in fuseki with DataAccessCtl and SecurityContext. Well, there is not an explicit DatasetGraph view for a certain agent, but the SecurityContext corresponds to an agent's permissions and there is `SecurityContext.createQueryExecution(Query, DatasetGraph)` to evaluate a query on a dataset w.r.t. a set of permissions.
For security over functions and property functions it seems that what would be needed is an extended or new SecurityContext with additional allow/deny lists and a permission check at an appropriate place (probably in ARQ) upon function evaluation.
> But that only goes so far.
Hm, is there much more complexity to it?
So I think for the first release of _this_ PR the security of the cache management is not that critical but its likely to be eventually needed in practice.
> This particular case could be done by context+function registry and/or datasets with different setups over the same data.
Yes, that was what I was aiming for.
**W.r.t to fuseki modules:** We are already making use of https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/apache-jena-fuseki/fuseki-server (rather than https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/jena-fuseki-server/fuseki-main) which loads all jars from the `./extra` folder. Not sure why there are two scripts with seemingly similar purpose - is the latter becoming the new start script?
The criticized shade profile in the serviceenhancer module is there to create the the self-contained jar that can be placed into the `extras` folder and thus on fuseki's classpath. So I think the main question here is which jena maven module should I use an example setup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939540847
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
Hm, do you know if it's possible to generate these warnings also with a cli `mvn` invocation?
At least adding `-Pdev` or `-Dmaven.compiler.showWarnings` does not generate them.
I can warnings with IntelliJ analyze code though - will go through them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939569153
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
In my Eclipse 2022-06 I had to manually enable the mentioned warnings - they now seem to be on ignore by default (at least I clicked on restore defaults and they remained on ignore) - I am pretty sure they were enabled in the past though.
Anyway, all the mentioned ones plus some additional ones should now be fixed and I squashed - can you confirm?
- [x] Check deployment to our test bed to make sure that my changes didn't break the fuseki setup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939525193
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
I am still get 36 warning from Eclipse:
* Javadoc : `@return` without text after it.
* try-with-resource
* Missing `@Override`
* Unnecessary casts
Eclispe or IntelliJ will pick these up.
When people start working with the Jena codebase, it's nice to give them a reasonable clean view.
As Release Manager, it is also helpful to have a clean view so I can see when new things arise.
```
Javadoc: Description expected after @return BatchQueryRewriter.java line 129
Javadoc: Description expected after @return IteratorFactoryWithBuffer.java line 96
Javadoc: Description expected after @return Ref.java line 73
Javadoc: Description expected after @return ServiceOpts.java line 118
Javadoc: Description expected after @return Slice.java line 57
Javadoc: Description expected after @return Slice.java line 90
Javadoc: Description expected after @return VarScopeUtils.java line 105
Resource 'disposable' should be managed by try-with-resource AsyncClaimingCacheImplGuava.java line 378 Java Problem
Resource 'reclaim' should be managed by try-with-resource AsyncClaimingCacheImplGuava.java line 385
Resource 'ref' should be managed by try-with-resource AsyncClaimingCacheImplGuava.java line 376
The method claim(K) of type AsyncClaimingCacheImplGuava<K,V> should be tagged with @Override since it actually overrides a superinterface method AsyncClaimingCacheImplGuava.java line 162
The method close() of type SliceAccessor<A> should be tagged with @Override since it actually overrides a superinterface method SliceAccessor.java line 108
The method createSubIterator(boolean) of type IteratorFactoryWithBuffer<T,I>.SubIteratorImpl should be tagged with @Override since it actually overrides a superinterface method IteratorFactoryWithBuffer.java line 135
The method exec(OpService) of type OpServiceExecutorImpl should be tagged with @Override since it actually overrides a superinterface method OpServiceExecutorImpl.java line 51
The method getArrayOps() of type BufferLike<A> should be tagged with @Override since it actually overrides a superinterface method BufferLike.java line 33
The method getDelegate() of type RefDelegateBase<T,R> should be tagged with @Override since it actually overrides a superinterface method RefDelegateBase.java line 31
The method getFailedRanges() of type SliceMetaDataImpl should be tagged with @Override since it actually overrides a superinterface method SliceMetaDataImpl.java line 70
The method getItems() of type BatchImpl<K,T> should be tagged with @Override since it actually overrides a superinterface method BatchImpl.java line 76
The method getLoadedRanges() of type SliceMetaDataImpl should be tagged with @Override since it actually overrides a superinterface method SliceMetaDataImpl.java line 62
The method getMaximumKnownSize() of type SliceMetaDataImpl should be tagged with @Override since it actually overrides a superinterface method SliceMetaDataImpl.java line 78
The method getMinimumKnownSize() of type SliceMetaDataImpl should be tagged with @Override since it actually overrides a superinterface method SliceMetaDataImpl.java line 74
The method getNextValidIndex() of type BatchImpl<K,T> should be tagged with @Override since it actually overrides a superinterface method BatchImpl.java line 68
The method getRanges() of type RangeBufferImpl<A> should be tagged with @Override since it actually overrides a superinterface method RangeBufferImpl.java line 90
The method isEmpty() of type BatchImpl<K,T> should be tagged with @Override since it actually overrides a superinterface method BatchImpl.java line 80
The method newSliceAccessor() of type SliceWithPages<T> should be tagged with @Override since it actually overrides a superinterface method SliceWithPages.java line 31
The method put(K, T) of type BatchImpl<K,T> should be tagged with @Override since it actually overrides a superinterface method BatchImpl.java line 58
The method size() of type BatchImpl<K,T> should be tagged with @Override since it actually overrides a superinterface method BatchImpl.java line 84
The method start() of type ServiceEnhancerInit should be tagged with @Override since it actually overrides a superinterface method ServiceEnhancerInit.java line 58
The method unsafeRead(A, int, long, int) of type SliceAccessorImpl<A> should be tagged with @Override since it actually overrides a superinterface method SliceAccessorImpl.java line 286
Unnecessary cast from A to A BufferOverArray.java line 50
Unnecessary cast from int to long PageUtils.java line 44
Unnecessary cast from K to K AsyncClaimingCacheImplGuava.java line 307
Unnecessary cast from long to long RangeBufferImpl.java line 83
Unnecessary cast from Ref to Ref<T> RefImpl.java line 162
Unnecessary cast from V to V AsyncClaimingCacheImplGuava.java line 307
Unnecessary cast from V to V AsyncClaimingCacheImplGuava.java line 324
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r936697620
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
`mvn clean install` and `mvn javadoc:javadoc` succeeds without any warnings at least for me.
I removed many adjacent blank lines and commented code fragments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r936697620
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
`mvn clean install` and `mvn javadoc:javadoc` succeeds without any warnings at least for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1154191966
I'm unsubscribing from this PR due to the stream of notifications - can we keep the discussion on the issue please, particularly of the core changes and bulk in the extra module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1166309676
Right - it does not have to be all commits. When there is a reference though, it's nice to use the same syntax everywhere - helps when looking through many items and helps scripts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r929917259
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
I am not totally sure what setup you prefer here - the poms of which modules should be adjusted and how?
The current functionality of the serviceenhancer extension works with fuseki without the need for a fuseki module - the service loader of the serviceenhancer registers an assembler that can be used to construct a dataset that can be exposed via a fuseki service.
That means that just adding the serviceenhancer plugin as a dependency to fuseki directly would so far be sufficient to enable configuration via assembler.
There is another issue however which I am not sure how to tackle best: Execution of the `<java:org.apache.jena.sparql.service.enhancer.function.cacheInvalidate>` function should be restricted to only authorized users. Otherwise malicious agents could disrupt a service by constantly invalidation all caches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r934591931
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
Review Comment:
This was just a copy&paste from the querybuilder pom which I used as a base. It's removed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r936696287
##########
jena-extras/jena-serviceenhancer/README.md:
##########
@@ -0,0 +1,259 @@
+# Service Enhancer Plugin
+
+This plugin extends the functionality of the SERVICE clause with:
+
+- Bulk requests
+- Correlated joins also known as lateral joins
+- Caching
+
+As a fundamental principle, a request making use of cache and bulk should return the exact same result as if
+those settings were omitted. As a consequence runtime result set size recognization (RRR) is employed to reveal hidden
+result set limits and ensure that always only the appropriate amount of data is yeld from the caches.
Review Comment:
Fixed and doc is now a PR at jena-site
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1203830899
@afs Are there any unaddressed issues remaining? From my side am currently doing the last testing and minor fixes/cleanup, but overall the test cases as well as our fuseki setup still seems to work as expected. So it looks like I can do a final squash today.
One question is how people should get the plugin into fuseki - right now they'd have to build the plugin with the `bundle` profile.
Should there be a separate maven module that creates the fuseki bundle which is published to maven central?
Currently, because the plugin has no other dependencies besides it should work to use the module's published jar file directly.
But this will break once any third-party dependency is added so the mechanism is brittle and should not be relied upon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1201274443
> `urn:base` is an illegal URN and causes a parser warning.
Right, I now use `urn:example:base` which I hope is valid. I am making an explicit reference in the example with `model.getResource("urn:example:root")` so a relative IRI or blank node is not really helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1201495883
> > Should I move all IRIs introduced by the plugin to that namespace? - e.g. `se:DatasetServiceEnhancer` instead of `ja:DatasetServiceEnhancer`? This would make it more self-contained.
>
> Yes, that would be best.
Done (a bit earlier today).
> It's nearly release time! Final confirmation ...
I think I can address the remaining open issues until tonight followed by a final round of testing tomorrow (which hopefully does not reveal any serious regressions introduced by the cleanup).
We have used/tested the module internally for our use cases for a while now and all encountered issues were ironed out - so it should be ready for an experimental release for gathering additional feedback and possibly also issue reports.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs merged pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs merged PR #1315:
URL: https://github.com/apache/jena/pull/1315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939540847
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
Hm, do you know if it's possible to generate these warnings also with a cli `mvn` invocation?
At least adding `-Pdev` or `-Dmaven.compiler.showWarnings` does not generate them.
I can see warnings with IntelliJ analyze code though - will go through them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939569153
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
In my Eclipse 2022-06 I had to manually enable the mentioned warnings - they now seem to be on ignore by default (at least I clicked on restore defaults and they remained on ignore) - I am pretty sure they were enabled in the past though.
Anyway, all the mentioned ones plus some additional ones should now be fixed and I squashed - can you confirm?
I am currently re-deploying to our test bed to make sure that my changes don't break the fuseki setup.
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
In my Eclipse 2022-06 I had to manually enable the mentioned warnings - they now seem to be on ignore by default (at least I clicked on restore defaults and they remained on ignore) - I am pretty sure they were enabled in the past though.
Anyway, all the mentioned ones plus some additional ones should now be fixed and I squashed - can you confirm?
I am currently re-deploying to our test bed to make sure that my changes didn't break the fuseki setup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1179191892
When you get a moment (!!), could you rebase this on current main? Thanks.
We're moving towards Jena 4.6.0 end this month -ish. (July)
Is the hope to get this code in that release? If it is all changes under jena-extras, it can go out with a suitable disclaimer "subject to change". Documentation can, if necessary, follow later (but a placeholder at release time would be nice for a link in the release announcement.
This is not a "must" - if it is not ready, then fine, whatever works for you and whatever plans you have coming up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on pull request #1315: GH-1314 Bulk retrieval with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1127991860
> It's currently a draft
Please set to "draft"!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1146247350
Yes, I will split the PR up. I started testing the cache system (and thus the plugin system) on some of our use cases and I am currently rooting out the most obvious issues. Moving the cache to `jena-extras` once it is sufficiently stable sound very good to me.
As an estimate, I am targeting a PR for the extension system in two weeks (next week for more testing, and the week after for factoring things out).
> the interactions a query that combines local patterns with SERVICE: "SERVICE - local -SERVICE"
Just to clarify the scope of this work: At this stage, I am not building a query optimizer that tries to re-order the algebra. I appreciate the "straight-forwardness" of SPARQL basic federation which works without statistics over the data in remote endpoints.
The goal is to enable caching of the results of the service clause's graph pattern w.r.t. to each individual input binding (which may be the root/identity binding). Because of the bulk feature, multiple input bindings may be combined into a batch request but the system disentangles this on a per-binding level again - so if the same service pattern is joined with the same binding it will cause a cache hit - regardless whether that input binding originated from e.g. a sub query or a values clause.
Because of this design goal, the bulk retrieval and the caching system cannot be completely independent from each other.
The goal of the cache is to allow for *manual* marking expensive parts of a query - e.g. parts that make use of ordering or aggregation. In Apache Spark terms it would be quite akin to an `rdd.cache()` operation.
Conversely, the goal is NOT to automatically decide on expensive parts (such as by looking at the query history and trying to make smart decision).
For example, one use case is to have a UI with a search box that allows filtering e.g. the available predicates.
With the cache, it becomes possible to filter over the cached result set - without having to compute it over and over again or the need to cache the result with custom logic in. e.g. java script.
```sparql
SELECT * {
SERVICE <cache:http://my.endpoint/sparql> { SELECT DISTINCT ?p { ?s ?p ?o } ORDER BY ?p }
FILTER(CONTAINS(STR(?p), 'rdf'))
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r930165210
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
ARQ does not have the SQL security model.
SQL procedures aren't queries and aren't functions - they are any SQL statements, they have their defined execution semantics as to order and they can have side effects.
This particular case could be done by context+function registry and/or datasets with different setups over the same data. But that only goes so far.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931224743
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
**About security**: I was exactly thinking about having some factory method that creates appropriate graph views from a base dataset, a security model and a given user (actor). This is pretty much what is being done in fuseki with DataAccessCtl and SecurityContext. Well, there is not an explicit DatasetGraph view for a certain agent, but the SecurityContext corresponds to an agent's permissions and there is `SecurityContext.createQueryExecution(Query, DatasetGraph)` to evaluate a query on a dataset w.r.t. a set of permissions.
For security over functions and property functions it seems that what would be needed is an extended or new SecurityContext with additional allow/deny lists and a permission check at an appropriate place (probably in ARQ) upon function evaluation.
> But that only goes so far.
Hm, is there much more complexity to it?
So I think for the first release of _this_ PR the security of the cache management is not that critical but its likely to be eventually needed in practice.
> This particular case could be done by context+function registry and/or datasets with different setups over the same data.
Yes, that was what I was aiming for.
**W.r.t to fuseki modules:** We are already making use of https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/apache-jena-fuseki/fuseki-server (rather than https://github.com/afs/jena/blob/fuseki-main-script/jena-fuseki2/jena-fuseki-server/fuseki-main) which loads all jars from the `./extra` folder. Not sure why there are two scripts with seemingly similar purpose - is the latter becoming the new start script?
The criticized shade profile in the serviceenhancer module is there to create the the self-contained jar that can be placed into the `extras` folder and thus on fuseki's classpath. So I think the main question here is which jena maven module should I use an example setup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931239846
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> Not sure why there are two scripts with seemingly similar purpose
Different Fuseki's : webapp and main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] afs commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
afs commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r931355751
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> is the latter becoming the new start script?
Script `apache-jena-fuseki/fuseki-server` runs `jena-fuseki-fulljar` (not `jena-fuseki-server`) renamed into `apache-jena-fuseki` as `fuseki-server.jar`.
The goal is replace `jena-fuseki-fulljar`, a webapp system, in favour of Fuseki/Main together with modules for the UI and admin functionality.
`jena-fuseki-war` remains but it will be the only webapp based form.
Teh webapp version does not support FusekiModules. Only Fuseki/Main has that extension mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r935479298
##########
jena-extras/jena-serviceenhancer/src/test/java/org/apache/jena/sparql/service/enhancer/impl/TS_ResultSetLimits.java:
##########
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.service.enhancer.impl;
+
+import java.util.IdentityHashMap;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+import org.apache.jena.query.Dataset;
+import org.apache.jena.query.DatasetFactory;
+import org.apache.jena.query.Query;
+import org.apache.jena.query.QueryExecution;
+import org.apache.jena.query.QueryFactory;
+import org.apache.jena.query.ResultSetFactory;
+import org.apache.jena.query.ResultSetFormatter;
+import org.apache.jena.query.ResultSetRewindable;
+import org.apache.jena.rdf.model.Model;
+import org.apache.jena.rdf.model.ModelFactory;
+import org.apache.jena.rdf.model.Property;
+import org.apache.jena.rdf.model.Resource;
+import org.apache.jena.rdf.model.ResourceFactory;
+import org.apache.jena.sparql.engine.iterator.QueryIterSlice;
+import org.apache.jena.sparql.service.ServiceExecutorRegistry;
+import org.apache.jena.sparql.service.enhancer.init.InitServiceEnhancer;
+import org.apache.jena.sparql.util.Context;
+import org.apache.jena.vocabulary.RDF;
+import org.apache.jena.vocabulary.RDFS;
+import org.junit.Assert;
+import org.junit.Test;
+
+public abstract class TS_ResultSetLimits {
Review Comment:
I updated the test cases to start with `[Abstract]TestServiceEnhancer...`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932396699
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> create a separate endpoint API that can be protected.
The following setup with now works with Fuseki. TestDatasetAssemblerServiceEnhancer now also includes tests for that setup with expected success / failure result.
```turtle
<#testService> rdf:type fuseki:Service; fuseki:name "test"; fuseki:dataset <#testDs> .
<#testServiceAdmin> rdf:type fuseki:Service; fuseki:name "testAdmin"; fuseki:dataset <#testDsAdmin> .
<#testDs> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> .
<#testDsAdmin> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> ; ja:enableMgmt true .
<#testDsCore> a ja:MemoryDataset .
```
If `ja:enableMgmt` is true, then the assembler for DatasetServiceEnhancer creates a DatasetGraphWrapper
with a copy of the base dataset's context with an additional symEnableMgmt attribute set to true.
The `afn:cacheRm` method checks for the presence of the flag and fails if it is absent.
(I renamed cacheList and cacheInvalidate to cacheLs and cacheRm as probably its easier to remember when one is already familiar with linux file system commands or docker - e.g. `docker image|volume ls`, `rm foo`, etc.)
What namespaces do you suggest? Right now I just added to the ja / afn /apf namespaces which avoids having to deal with more namespaces but maybe you'd prefer a plugin specific namespace such as `<http[s]://jena.apache.org/extra/service-enhancer/{function/pfunction/assembler}#>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on PR #1315:
URL: https://github.com/apache/jena/pull/1315#issuecomment-1201033674
> README.md in the code base isn't the easiest or natural place for users.
I opened a PR for jena-site with the documentation: https://github.com/apache/jena-site/pull/113
I can basically replace the readme in the main repo with a link to jena-site.
For the namespace I would prefer `se: http://jena.apache.org/service-enhancer#` with a dash (`-`) but if you insist on underscore `_` it's also fine with me. I think there is no need to make things more complex by introducing further namespaces for functions / property functions / assembers.
Should I move all IRIs introduced by the plugin to that namespace? - e.g. `se:DatasetServiceEnhancer` instead of `ja:DatasetServiceEnhancer`? This would make it more self-contained.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r932396699
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>jena-serviceenhancer</artifactId>
+ <name>Apache Jena - Extras - Service Enhancer</name>
+ <description>A plugin that extends the sparql SERVICE clauses with bulk requests, lateral joins and advanced result set caching.</description>
+
+ <parent>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>jena-extras</artifactId>
+ <relativePath>..</relativePath>
+ <version>4.6.0-SNAPSHOT</version>
+ </parent>
+
+ <properties>
+ <automatic.module.name>org.apache.jena.serviceplugins</automatic.module.name>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.jena</groupId>
+ <artifactId>apache-jena-libs</artifactId>
+ <type>pom</type>
+ </dependency>
+
+ <dependency>
+ <groupId>com.github.ben-manes.caffeine</groupId>
+ <artifactId>caffeine</artifactId>
+ <version>3.1.1</version>
+ </dependency>
+
+ <!--
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-io-buffers</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.aksw.commons</groupId>
+ <artifactId>aksw-commons-claimingcache</artifactId>
+ <version>0.9.5-SNAPSHOT</version>
+ </dependency>
+ -->
+
+ <dependency>
+ <groupId>org.xenei</groupId>
+ <artifactId>junit-contracts</artifactId>
+ <scope>test</scope>
+ <exclusions>
+ <exclusion>
+ <artifactId>commons-cli</artifactId>
+ <groupId>commons-cli</groupId>
+ </exclusion>
+ <exclusion>
+ <artifactId>commons-logging</artifactId>
+ <groupId>commons-logging</groupId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifestEntries>
+ <Automatic-Module-Name>${automatic.module.name}</Automatic-Module-Name>
+ </manifestEntries>
+ </archive>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-source-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>attach-sources</id>
+ <!-- <phase>package</phase> package is the default -->
+ <goals>
+ <goal>jar-no-fork</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+ <!-- Profile to build a "plugin jar bundle" that can be loaded with fuseki or any other jena-based app -->
Review Comment:
> create a separate endpoint API that can be protected.
The following setup with now works with Fuseki. TestDatasetAssemblerServiceEnhancer now also includes tests whether t
```turtle
<#testService> rdf:type fuseki:Service;
fuseki:name "test";
fuseki:dataset <#testDs> .
<#testServiceAdmin> rdf:type fuseki:Service;
fuseki:name "testAdmin";
fuseki:dataset <#testDsAdmin> .
<#testDs> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> .
<#testDsAdmin> a ja:DatasetServiceEnhancer ; ja:baseDataset <#testDsCore> ; ja:enableMgmt true .
<#testDsCore> a ja:MemoryDataset .
```
If `ja:enableMgmt` is true, then the assembler for DatasetServiceEnhancer creates a DatasetGraphWrapper
with a copy of the base dataset's context with an additional symEnableMgmt attribute set to true.
The `afn:cacheRm` method checks for the presence of the flag and fails if it is absent.
(I renamed cacheList and cacheInvalidate to cacheLs and cacheRm as probably its easier to remember when one is already familiar with linux file system commands or docker - e.g. `docker image|volume ls`, `rm foo`, etc.)
What namespaces do you suggest? Right now I just added to the ja / afn /apf namespaces which avoids having to deal with more namespaces but maybe you'd prefer a plugin specific namespace such as `<http[s]://jena.apache.org/extra/service-enhancer/{function/pfunction/assembler}#>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org
[GitHub] [jena] Aklakan commented on a diff in pull request #1315: GH-1314 Bulk retrieval and caching with SERVICE clause
Posted by GitBox <gi...@apache.org>.
Aklakan commented on code in PR #1315:
URL: https://github.com/apache/jena/pull/1315#discussion_r939684734
##########
jena-extras/jena-serviceenhancer/pom.xml:
##########
@@ -0,0 +1,185 @@
+<?xml version="1.0" encoding="UTF-8"?>
Review Comment:
I noticed those potential resource leaks warnings - and I wonder if there is any meaningful design pattern to make those go away:
Essentially these methods allocate resources which are used in the set up of the query iterators. Because of the streaming approach they can only be closed once the iterator is closed. Wrapping them with try-with-resources does not make sense there.
I tried suppressing some of them but it's kind of a catch-22: The suppressions themselves generate warnings depending on the IDE compiler settings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org