Demo of Fuseki in large-scale geo data production

[Nicholas Car <ni...@kurrawong.net>]

Dear Jena users,

I'm happy to announce that the Australian government now has a series of large spatial datasets delivered as Linked Data via web APIs that use Fuseki as the back end in production. This is the firts operation phase of a long-term semantic spatial data layer for Australia.

The data is significant: authoritative census counting geometries, hydrological catchment areas and so on. The census spatial feature collections are online at:

https://asgs.linked.fsdf.org.au/dataset/asgsed3/collections

A single census region: https://asgs.linked.fsdf.org.au/dataset/asgsed3/collections/SA1/items/10102100710

The largest dataset, the Geocoded National Address file, has 14.5M addresses (all Australian street addresses) with perhaps 50 triples each running well on a single GeoFuseki instance with GeoSPARQL indexing:

Feature Collections: https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections
Address FC: https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses
Addresses: https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses/items
An Address object: https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses/items/GANSW718188625

The data from these APIs provide the points-of-truth for national spatial dataset object identity, e.g. the address above with the persistent IRI of:

https://linked.data.gov.au/dataset/gnaf/address/GANSW718188625 

We are also testing the use of all the various datasets combined in a Digital Atlas of Australia Analytics Platform. That will see complex SPARQL qrieries posed against all the data with geospatial and text filtering, for which we will soon implement Fuseki Lucene text indexes on top of the GeoSPARQL indexes.

We have spent a fair bit of time getting RDF generation from non-RDF sources right, running RIOT tooling to check and load RDF data and to build various indexes, mostly in AWS cloud infrastructure. We thank:

* Stian Soiland-Reyes
* Zazuko
* Finland's Semantic Computing Research Group 

...for their Fuseki containerisation that has made this relatively easy. 

Of course, we thank Andy Seabourne and the Apache Foundation for Jena/Fuseki itself!

Regards,

Nicholas Car
nick@kurrawong.net

David Habgood
dcchabgood@gmail.com

Re: Demo of Fuseki in large-scale geo data production

[Luis Enrique Ramos García <lu...@googlemail.com.INVALID>]

Congratulations!!!

great news.


Luis

El vie, 7 oct 2022 a las 8:19, Bruno Kinoshita (<ki...@apache.org>)
escribió:

> Hi Nick,
>
> It is looking really good! I wish (and hope) someone would do the same for
> NZ data (digitalNZ/Stats/at/etc). Looks like you used GeoSPARQL too, it is
> the first time I accessed a service with it (that was not just
> dev/research), I think.
>
> Great work!
>
> -Bruno
>
> On Fri, 7 Oct 2022 at 18:39, Nicholas Car <ni...@kurrawong.net> wrote:
>
> > Dear Jena users,
> >
> > I'm happy to announce that the Australian government now has a series of
> > large spatial datasets delivered as Linked Data via web APIs that use
> > Fuseki as the back end in production. This is the firts operation phase
> of
> > a long-term semantic spatial data layer for Australia.
> >
> > The data is significant: authoritative census counting geometries,
> > hydrological catchment areas and so on. The census spatial feature
> > collections are online at:
> >
> > https://asgs.linked.fsdf.org.au/dataset/asgsed3/collections
> >
> > A single census region:
> >
> https://asgs.linked.fsdf.org.au/dataset/asgsed3/collections/SA1/items/10102100710
> >
> > The largest dataset, the Geocoded National Address file, has 14.5M
> > addresses (all Australian street addresses) with perhaps 50 triples each
> > running well on a single GeoFuseki instance with GeoSPARQL indexing:
> >
> > Feature Collections:
> > https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections
> > Address FC:
> > https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses
> > Addresses:
> > https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses/items
> > An Address object:
> >
> https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses/items/GANSW718188625
> >
> > The data from these APIs provide the points-of-truth for national spatial
> > dataset object identity, e.g. the address above with the persistent IRI
> of:
> >
> > https://linked.data.gov.au/dataset/gnaf/address/GANSW718188625
> >
> > We are also testing the use of all the various datasets combined in a
> > Digital Atlas of Australia Analytics Platform. That will see complex
> SPARQL
> > qrieries posed against all the data with geospatial and text filtering,
> for
> > which we will soon implement Fuseki Lucene text indexes on top of the
> > GeoSPARQL indexes.
> >
> > We have spent a fair bit of time getting RDF generation from non-RDF
> > sources right, running RIOT tooling to check and load RDF data and to
> build
> > various indexes, mostly in AWS cloud infrastructure. We thank:
> >
> > * Stian Soiland-Reyes
> > * Zazuko
> > * Finland's Semantic Computing Research Group
> >
> > ...for their Fuseki containerisation that has made this relatively easy.
> >
> > Of course, we thank Andy Seabourne and the Apache Foundation for
> > Jena/Fuseki itself!
> >
> > Regards,
> >
> > Nicholas Car
> > nick@kurrawong.net
> >
> > David Habgood
> > dcchabgood@gmail.com
> >
>

Re: Demo of Fuseki in large-scale geo data production

[Bruno Kinoshita <ki...@apache.org>]

Hi Nick,

It is looking really good! I wish (and hope) someone would do the same for
NZ data (digitalNZ/Stats/at/etc). Looks like you used GeoSPARQL too, it is
the first time I accessed a service with it (that was not just
dev/research), I think.

Great work!

-Bruno

On Fri, 7 Oct 2022 at 18:39, Nicholas Car <ni...@kurrawong.net> wrote:

> Dear Jena users,
>
> I'm happy to announce that the Australian government now has a series of
> large spatial datasets delivered as Linked Data via web APIs that use
> Fuseki as the back end in production. This is the firts operation phase of
> a long-term semantic spatial data layer for Australia.
>
> The data is significant: authoritative census counting geometries,
> hydrological catchment areas and so on. The census spatial feature
> collections are online at:
>
> https://asgs.linked.fsdf.org.au/dataset/asgsed3/collections
>
> A single census region:
> https://asgs.linked.fsdf.org.au/dataset/asgsed3/collections/SA1/items/10102100710
>
> The largest dataset, the Geocoded National Address file, has 14.5M
> addresses (all Australian street addresses) with perhaps 50 triples each
> running well on a single GeoFuseki instance with GeoSPARQL indexing:
>
> Feature Collections:
> https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections
> Address FC:
> https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses
> Addresses:
> https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses/items
> An Address object:
> https://gnaf.linked.fsdf.org.au/dataset/gnaf/collections/addresses/items/GANSW718188625
>
> The data from these APIs provide the points-of-truth for national spatial
> dataset object identity, e.g. the address above with the persistent IRI of:
>
> https://linked.data.gov.au/dataset/gnaf/address/GANSW718188625
>
> We are also testing the use of all the various datasets combined in a
> Digital Atlas of Australia Analytics Platform. That will see complex SPARQL
> qrieries posed against all the data with geospatial and text filtering, for
> which we will soon implement Fuseki Lucene text indexes on top of the
> GeoSPARQL indexes.
>
> We have spent a fair bit of time getting RDF generation from non-RDF
> sources right, running RIOT tooling to check and load RDF data and to build
> various indexes, mostly in AWS cloud infrastructure. We thank:
>
> * Stian Soiland-Reyes
> * Zazuko
> * Finland's Semantic Computing Research Group
>
> ...for their Fuseki containerisation that has made this relatively easy.
>
> Of course, we thank Andy Seabourne and the Apache Foundation for
> Jena/Fuseki itself!
>
> Regards,
>
> Nicholas Car
> nick@kurrawong.net
>
> David Habgood
> dcchabgood@gmail.com
>