You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Andy Seaborne <an...@apache.org> on 2022/05/22 21:35:31 UTC

[Preview] Fuseki update from kafka

This is a preview of a potential contribution to Apache Jena.

It is a Fuseki Module [*] that connects an Apache Kafka topic to a 
Fuseki dataset, so that the dataset can be updated by Kafka events.

https://github.com/Telicent-io/jena-fuseki-kafka/

The connector tracks the state of the topic and the dataset. It records 
the last event offset successfully used to update the datasets and 
updates as new events come in.

Events sent to the topic when the Fuseki server is not running are 
picked up and applied to bring the dataset up-to-date when the server 
starts or restarts.

It means that the dataset can be built from the topic, by starting with 
empty dataset and no prior tracked topic state.

It will also work with RDF Patch.

Current status:

It works, supporting sending data (triple or quads) for inclusion in a 
dataset and also SPARQL Update requests. These can be mixed within a 
kafka topic.

This work was done at Telicent.

It needs refining for different Kafka setups - it has been deployed in 
the Telicent CORE platform. Currently, you need to build it locally. See 
README.md.

     Andy

[*] An additional jar to drop into a Fuseki Main server installation. A 
Fuseki Module hooks into the server build cycle can modify the server 
being built. Prototype fuseki-main bash script available.

https://jena.apache.org/documentation/fuseki2/fuseki-modules