Re: [Apache Fuseki] Limits of Apache Fuseki Triple Store

[Andy Seaborne <an...@apache.org>]

On 25/01/2021 07:02, Marco Franke wrote:
> Dear developers,
> 
> we use the Apache Fuseki Triple Store to store sensory data. Up to now, 
> it works pretty well.
> 
> The challenge is that we have to import a lot of data in the upcoming month.
> 
> At the moment, we have stored already ca. 500 000 triples and everything 
> is fine.

In memory, 10's of millions compact and in-memory isn't something that 
is specially optimized but the bounds are RAM not design. e.g. the 
default heap size (-Xmx in the scripts) for Fuseki is quite low.

See Pauls' reply for when using TDB.

> My question is whether there is a limitation of the amount of triples to 
> be stored and queried?

Internal design limitations are much higher.

> My second question is how to download the data again.
> 
> I know there is the …./get endpoint for each endpoint. Does this 
> endpoint works for huge datasets, too?

Yes - it streams and it is transaction so it reads consistent snapshot 
and does not require all the data in memory at once.

See also the /$/backup operation which writes a quads file in the 
background to the local disk (so it does not go over the network and 
does not require the client to hang around).

     Andy

> 
> Thanks in Advance,
> 
> Kind regards,
> 
> Marco Franke
> 
> __
> 
> *i.A. Dipl. -Inf. Marco Franke***
> 
> Wissenschaftlicher Mitarbeiter in 2.2
> 
> *BIBA**- Bremer Institut für Produktion und Logistik GmbH*
> 
> Informations- und kommunikationstechnische Anwendungen in der Produktion
> 
> Prof. Dr.-Ing. Klaus-Dieter Thoben
> 
> Raum 1490
> Tel.: +49 (0)421 218-50089
> 
> Fax: +49 (0)421 218-50007
> 
> fma@biba.uni-bremen.de <ma...@biba.uni-bremen.de>
> 
> In-2C-28px-R 
> <https://www.linkedin.com/company/biba.uni-bremen.de>apple-touch-icon-180x180 
> <https://www.researchgate.net/institution/BIBA-Bremer_Institut_fuer_Produktion_und_Logistik>FB-f-Logo__blue_29 
> <https://www.facebook.com/BIBA.Produktion.Logistik>YouTube-social-icon_red_48px 
> <http://www.youtube.com/channel/UCieF5Uq5Qix9XZAZuYhTQhg>
> 
> BIBA – Bremer Institut für Produktion und Logistik GmbH
> 
> Postanschrift: Postfach P.O.B. 33 05 60 · 28335 Bremen / Germany
> 
> Geschäftssitz: Hochschulring 20 · 28359 Bremen / Germany
> 
> USt-ID: DE814890109 · Amtsgericht Bremen HRB 24505 HB
> 
> Tel: +49 (0)421/218-50000 · Fax: +49 (0)421/218-50031
> 
> E-Mail: info@biba.uni-bremen.de <ma...@biba.uni-bremen.de> 
> · Internet: www.biba.uni-bremen.de <http://www.biba.uni-bremen.de/>
> 
> Geschäftsführer: Prof. Dr.-Ing. M. Freitag, Prof. Dr.-Ing. K.-D. Thoben, 
> O. Simon
>