You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jena.apache.org by "decdocuteam (via GitHub)" <gi...@apache.org> on 2026/01/06 07:59:41 UTC

[D] Difference between tdb2.tdbupdater and update endpoint [jena]

GitHub user decdocuteam created a discussion: Difference between tdb2.tdbupdater and update endpoint

We are experiencing performance problem issues on update of large td2 dataset. We see huge difference on update response time on some of our resources that we cannot explain. We use tdb2.tdbupdater to upload the same triples and here the speed is normal. What is the difference between tdb2.tdbupdater and the update endpoint? We make sure that there was not others update or query during the update call. So it is hard for us to understand where the problem is.

I must add that updating on an empty tdb2 dataset is fast. So it is not a problem with the triples we try to insert. What we don't understand is why some triples are fast to update and others not.

thanks for the help
Vincent

GitHub link: https://github.com/apache/jena/discussions/3684

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
GitHub user afs added a comment to the discussion: Difference between tdb2.tdbupdater and update endpoint

Hi @decdocuteam,

I don't see any particular area that wouldd trigger the situation - may I ask for some more detail (below) so That I understand what's going on.

> doing the update through the Fedora Camel toolbox 

I don't know that that way of making updates. For a test would you be able to send an update dirct to the Fuseki endpoint and see whether it takes a similar length of time?

> What is the difference between tdb2.tdbupdate and the update endpoint?

If you have a tdb2.tdbupdate***r*** or tdb2.tdbupload then these is not from Jena - are they local customizations?

`tdbupdate` runs an update and should refuse to run if Fuseki is using the database. If `tdbupdate` is running wen Fuseki is (1) its not the same database, e.g. if it is a different `Data-` directory, or is in the top level TDB directory, or (2) maybe you have a container setup where the filesystem is mapped in which case two OS's are trying to manage the same DB - Jena tries to catch this but there are many ways to configure a container, and we may not be able to lock in all cases.

What is the storage configuration for the server? SSD? Disk? Virtualized remote disk?

Other than that, a SPARQL Update should be the same. What does an update look like?
Presumably the update is no using the Graph Store Protocol which can be used to send (i.e add) data.

What is the load on the server? While queries and updates are isolated and one update can happen in true parallel with queries, a heavy query load might affect the update.

Are updates made one at a time or are there many over lapping updates, all small?


GitHub link: https://github.com/apache/jena/discussions/3684#discussioncomment-15476107

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
GitHub user afs added a comment to the discussion: Difference between tdb2.tdbupdater and update endpoint

@decdocuteam How much data is being added on each update operation?

GitHub link: https://github.com/apache/jena/discussions/3684#discussioncomment-15421985

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "afs (via GitHub)" <gi...@apache.org>.
GitHub user afs edited a comment on the discussion: Difference between tdb2.tdbupdater and update endpoint

Hi @decdocuteam,

I don't see any particular area that wouldd trigger the situation - may I ask for some more detail (below) so That I understand what's going on.

> doing the update through the Fedora Camel toolbox 

I don't know that that way of making updates. For a test would you be able to send an update direct to the Fuseki endpoint (presumably this is Fuseki and not TDB2 used inside a different environment?) and see whether it takes a similar length of time?

> What is the difference between tdb2.tdbupdate and the update endpoint?

If you have a tdb2.tdbupdate***r*** or tdb2.tdbupload then these is not from Jena - are they local customizations?

`tdbupdate` runs an update and should refuse to run if Fuseki is using the database. If `tdbupdate` is running wen Fuseki is (1) its not the same database, e.g. if it is a different `Data-` directory, or is in the top level TDB directory, or (2) maybe you have a container setup where the filesystem is mapped in which case two OS's are trying to manage the same DB - Jena tries to catch this but there are many ways to configure a container, and we may not be able to lock in all cases.

What is the storage configuration for the server? SSD? Disk? Virtualized remote disk?

Other than that, a SPARQL Update should be the same. What does an update look like?
Presumably the update is no using the Graph Store Protocol which can be used to send (i.e add) data.

What is the load on the server? While queries and updates are isolated and one update can happen in true parallel with queries, a heavy query load might affect the update.

Are updates made one at a time or are there many over lapping updates, all small?


GitHub link: https://github.com/apache/jena/discussions/3684#discussioncomment-15476107

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "decdocuteam (via GitHub)" <gi...@apache.org>.
GitHub user decdocuteam closed the discussion with a comment: Difference between tdb2.tdbupdater and update endpoint

We found that the issue was the SPARQL request to do to delete before doing the actual update. Thanks for the help.

GitHub link: https://github.com/apache/jena/discussions/3684#discussioncomment-15614512

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "decdocuteam (via GitHub)" <gi...@apache.org>.
GitHub user decdocuteam added a comment to the discussion: Difference between tdb2.tdbupdater and update endpoint

for information I attach the stats of my tdb2 dataset.
[stats_dataset.txt](https://github.com/user-attachments/files/24453963/stats_dataset.txt)


GitHub link: https://github.com/apache/jena/discussions/3684#discussioncomment-15424914

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "decdocuteam (via GitHub)" <gi...@apache.org>.
GitHub user decdocuteam added a comment to the discussion: Difference between tdb2.tdbupdater and update endpoint

less than 100 triples so this is small update. We are using the Fedora stack and doing the update through the Fedora Camel toolbox using the update endpoint. This why we try to directly make the update with tdb2.tdbupload command to avoid all the overhead of the stack and have seen a big difference in execution time...

GitHub link: https://github.com/apache/jena/discussions/3684#discussioncomment-15422033

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org


Re: [D] Difference between tdb2.tdbupdater and update endpoint [jena]

Posted by "decdocuteam (via GitHub)" <gi...@apache.org>.
GitHub user decdocuteam closed a discussion: Difference between tdb2.tdbupdater and update endpoint

We are experiencing performance problem issues on update of large td2 dataset. We see huge difference on update response time on some of our resources that we cannot explain. We use tdb2.tdbupdater to upload the same triples and here the speed is normal. What is the difference between tdb2.tdbupdater and the update endpoint? We make sure that there was not others update or query during the update call. So it is hard for us to understand where the problem is.

I must add that updating on an empty tdb2 dataset is fast. So it is not a problem with the triples we try to insert. What we don't understand is why some triples are fast to update and others not.

thanks for the help
Vincent

GitHub link: https://github.com/apache/jena/discussions/3684

----
This is an automatically sent email for issues@jena.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@jena.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org