You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Andy Seaborne <an...@apache.org> on 2017/09/04 22:27:35 UTC
Contribution of TDB2
I'd like to offer TDB2 as a contribution to Apache Jena.
It is by no means "finished" (is anything software ever finished?) but
it is sufficiently there to talk about and kick the tyres; I'll send
notes about the technical state of TDB2 separately in a moment.
* It adds project overhead. The build is only a couple of minutes,
including upload time, (a lot of testing is done with in-memory
databases, like TDB1, which makes setup and teardown of tests much
faster). However, more stuff in the codebase is still more to get right
for a release.
* Support: while TDB2 works, it is unproven and rough round the edges
(unclear errors message, stacktraces rather than meaningful messages
etc) and questions/tickets may be quite hard to respond to (complex
situations; no track record of what's gone wrong before). Testing on MS
Windows or macOS has not been done.
* Integration: It is not stable enough to be used in mainstream Fuseki2
as the default persistent database. I see this as happening gradually.
There is a Fuseki+TDB2 jar built at the moment but you have to use
configuration files to setup the database.
https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
* Initial status: I suggest we label it "incoming" or some such - it
would not hold up a release for example, and not everything might be
fixed at the point of a Jena release. It shouldn't stop the main code
set that users rely on getting out in the usual cycle.
* Legal issues: the code has always been under the Apache License
(original work was done as part of grant from the UK gov which required
open source deliverables) but it would be simpler and cleaner for, e.g.
moving or fixing copyright/licence notices around, for there to be
software grants. The two parties involved are myself and Epimorphics
Ltd. Both parties are willing to submit the paperwork.
* A project vote. This is a new set of modules so a formal, explicit
vote seems worthwhile.
Thoughts, concerns, questions?
Andy
The code:
https://github.com/afs/mantis
(I can clean this up before integration as to what we agree on, so that
there is less noise and churn later).
Builds:
org.seaborne.mantis:tdb2
org.seaborne.mantis:fuseki-tdb2-server
Re: Contribution of TDB2
Posted by Dave Reynolds <da...@gmail.com>.
Sounds good to me, would be great to have this in the code base and
would be happy to +1 such a vote.
Dave
On 04/09/17 23:27, Andy Seaborne wrote:
> I'd like to offer TDB2 as a contribution to Apache Jena.
>
> It is by no means "finished" (is anything software ever finished?) but
> it is sufficiently there to talk about and kick the tyres; I'll send
> notes about the technical state of TDB2 separately in a moment.
>
> * It adds project overhead. The build is only a couple of minutes,
> including upload time, (a lot of testing is done with in-memory
> databases, like TDB1, which makes setup and teardown of tests much
> faster). However, more stuff in the codebase is still more to get right
> for a release.
>
> * Support: while TDB2 works, it is unproven and rough round the edges
> (unclear errors message, stacktraces rather than meaningful messages
> etc) and questions/tickets may be quite hard to respond to (complex
> situations; no track record of what's gone wrong before). Testing on MS
> Windows or macOS has not been done.
>
> * Integration: It is not stable enough to be used in mainstream Fuseki2
> as the default persistent database. I see this as happening gradually.
> There is a Fuseki+TDB2 jar built at the moment but you have to use
> configuration files to setup the database.
>
> https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
>
> * Initial status: I suggest we label it "incoming" or some such - it
> would not hold up a release for example, and not everything might be
> fixed at the point of a Jena release. It shouldn't stop the main code
> set that users rely on getting out in the usual cycle.
>
> * Legal issues: the code has always been under the Apache License
> (original work was done as part of grant from the UK gov which required
> open source deliverables) but it would be simpler and cleaner for, e.g.
> moving or fixing copyright/licence notices around, for there to be
> software grants. The two parties involved are myself and Epimorphics
> Ltd. Both parties are willing to submit the paperwork.
>
> * A project vote. This is a new set of modules so a formal, explicit
> vote seems worthwhile.
>
> Thoughts, concerns, questions?
>
> Andy
>
> The code:
> https://github.com/afs/mantis
> (I can clean this up before integration as to what we agree on, so that
> there is less noise and churn later).
>
> Builds:
> org.seaborne.mantis:tdb2
> org.seaborne.mantis:fuseki-tdb2-server
>
Re: Contribution of TDB2
Posted by Andy Seaborne <an...@apache.org>.
Having it used would be great. We can put it into Fuseki very soon,
and keep TDB1 as the default persistent storage in the UI.
and we could quickly have TDB2 in std-Fuseki
Actually, there is a build already:
http://central.maven.org/maven2/org/seaborne/mantis/fuseki-tdb2-server/
It should not affect query performance, in either direction. Its
doing the same thing in the same way.
Andy
On 7 September 2017 at 12:35, Osma Suominen <os...@helsinki.fi> wrote:
> Sounds very good to me! I'm especially interested in the space reclamation.
> Currently we do a manual rebuild of our TDB every 2 months or so, which is
> annoying. (Yes it could be automated, but...)
>
> Once this gets into a Fuseki release in usable form, I think we could use it
> on our Finto/Skosmos dev/test server.
>
> Is moving from TDB to TDB2 likely to affect SPARQL (read-only) query
> performance?
>
> -Osma
>
>
> Andy Seaborne kirjoitti 05.09.2017 klo 01:27:
>>
>> I'd like to offer TDB2 as a contribution to Apache Jena.
>>
>> It is by no means "finished" (is anything software ever finished?) but it
>> is sufficiently there to talk about and kick the tyres; I'll send notes
>> about the technical state of TDB2 separately in a moment.
>>
>> * It adds project overhead. The build is only a couple of minutes,
>> including upload time, (a lot of testing is done with in-memory databases,
>> like TDB1, which makes setup and teardown of tests much faster). However,
>> more stuff in the codebase is still more to get right for a release.
>>
>> * Support: while TDB2 works, it is unproven and rough round the edges
>> (unclear errors message, stacktraces rather than meaningful messages etc)
>> and questions/tickets may be quite hard to respond to (complex situations;
>> no track record of what's gone wrong before). Testing on MS Windows or
>> macOS has not been done.
>>
>> * Integration: It is not stable enough to be used in mainstream Fuseki2 as
>> the default persistent database. I see this as happening gradually. There
>> is a Fuseki+TDB2 jar built at the moment but you have to use configuration
>> files to setup the database.
>>
>> https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
>>
>> * Initial status: I suggest we label it "incoming" or some such - it
>> would not hold up a release for example, and not everything might be fixed
>> at the point of a Jena release. It shouldn't stop the main code set that
>> users rely on getting out in the usual cycle.
>>
>> * Legal issues: the code has always been under the Apache License
>> (original work was done as part of grant from the UK gov which required open
>> source deliverables) but it would be simpler and cleaner for, e.g. moving or
>> fixing copyright/licence notices around, for there to be software grants.
>> The two parties involved are myself and Epimorphics Ltd. Both parties are
>> willing to submit the paperwork.
>>
>> * A project vote. This is a new set of modules so a formal, explicit vote
>> seems worthwhile.
>>
>> Thoughts, concerns, questions?
>>
>> Andy
>>
>> The code:
>> https://github.com/afs/mantis
>> (I can clean this up before integration as to what we agree on, so that
>> there is less noise and churn later).
>>
>> Builds:
>> org.seaborne.mantis:tdb2
>> org.seaborne.mantis:fuseki-tdb2-server
>>
>
>
> --
> Osma Suominen
> D.Sc. (Tech), Information Systems Specialist
> National Library of Finland
> P.O. Box 26 (Kaikukatu 4)
> 00014 HELSINGIN YLIOPISTO
> Tel. +358 50 3199529
> osma.suominen@helsinki.fi
> http://www.nationallibrary.fi
Re: Contribution of TDB2
Posted by Osma Suominen <os...@helsinki.fi>.
Sounds very good to me! I'm especially interested in the space
reclamation. Currently we do a manual rebuild of our TDB every 2 months
or so, which is annoying. (Yes it could be automated, but...)
Once this gets into a Fuseki release in usable form, I think we could
use it on our Finto/Skosmos dev/test server.
Is moving from TDB to TDB2 likely to affect SPARQL (read-only) query
performance?
-Osma
Andy Seaborne kirjoitti 05.09.2017 klo 01:27:
> I'd like to offer TDB2 as a contribution to Apache Jena.
>
> It is by no means "finished" (is anything software ever finished?) but
> it is sufficiently there to talk about and kick the tyres; I'll send
> notes about the technical state of TDB2 separately in a moment.
>
> * It adds project overhead. The build is only a couple of minutes,
> including upload time, (a lot of testing is done with in-memory
> databases, like TDB1, which makes setup and teardown of tests much
> faster). However, more stuff in the codebase is still more to get right
> for a release.
>
> * Support: while TDB2 works, it is unproven and rough round the edges
> (unclear errors message, stacktraces rather than meaningful messages
> etc) and questions/tickets may be quite hard to respond to (complex
> situations; no track record of what's gone wrong before). Testing on MS
> Windows or macOS has not been done.
>
> * Integration: It is not stable enough to be used in mainstream Fuseki2
> as the default persistent database. I see this as happening gradually.
> There is a Fuseki+TDB2 jar built at the moment but you have to use
> configuration files to setup the database.
>
> https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
>
> * Initial status: I suggest we label it "incoming" or some such - it
> would not hold up a release for example, and not everything might be
> fixed at the point of a Jena release. It shouldn't stop the main code
> set that users rely on getting out in the usual cycle.
>
> * Legal issues: the code has always been under the Apache License
> (original work was done as part of grant from the UK gov which required
> open source deliverables) but it would be simpler and cleaner for, e.g.
> moving or fixing copyright/licence notices around, for there to be
> software grants. The two parties involved are myself and Epimorphics
> Ltd. Both parties are willing to submit the paperwork.
>
> * A project vote. This is a new set of modules so a formal, explicit
> vote seems worthwhile.
>
> Thoughts, concerns, questions?
>
> Andy
>
> The code:
> https://github.com/afs/mantis
> (I can clean this up before integration as to what we agree on, so that
> there is less noise and churn later).
>
> Builds:
> org.seaborne.mantis:tdb2
> org.seaborne.mantis:fuseki-tdb2-server
>
--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 26 (Kaikukatu 4)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.suominen@helsinki.fi
http://www.nationallibrary.fi
Re: Contribution of TDB2
Posted by Andy Seaborne <an...@apache.org>.
Notice that I'll merge TDB2 to master sometime soon.
https://github.com/apache/jena/tree/jena-tdb2
Andy
On 28/09/17 17:10, Andy Seaborne wrote:
> I think I've now done code ingestion, renaming and header formatting.
>
> So the large numbers of commits should be over!
>
> It is not wired into the build yet - it does build Jena artifacts now.
>
> Andy
>
>
> On 26/09/17 17:21, Andy Seaborne wrote:
>> We've VOTEd, and the software grants are now on-file.
>>
>> Next steps:
>>
>> I'll create a branch and move the code over as-is, old package names,
>> and all.
>>
>> If it's done as a pull, history will be preserved. The commits are by
>> my "@seaborne.org" email address, and also
>>
>> Natan Cox - fixes a typo in the documentation
>> https://github.com/afs/mantis/commit/6ca0caf
>>
>> Adam: code cleaning
>> https://github.com/afs/mantis/commit/38dda54
>>
>> next: prepare it - rename packages and put in under jena-db, in the
>> build but otherwise unconnected to the rest of Jena.
>>
>> then more review.
>>
>> Then merge to master. Until this point it's just kicking around in the
>> repo and not in the way of a release. The next release tick is end
>> October (ish).
>>
>> Andy
Re: Contribution of TDB2
Posted by Andy Seaborne <an...@apache.org>.
I think I've now done code ingestion, renaming and header formatting.
So the large numbers of commits should be over!
It is not wired into the build yet - it does build Jena artifacts now.
Andy
On 26/09/17 17:21, Andy Seaborne wrote:
> We've VOTEd, and the software grants are now on-file.
>
> Next steps:
>
> I'll create a branch and move the code over as-is, old package names,
> and all.
>
> If it's done as a pull, history will be preserved. The commits are by my
> "@seaborne.org" email address, and also
>
> Natan Cox - fixes a typo in the documentation
> https://github.com/afs/mantis/commit/6ca0caf
>
> Adam: code cleaning
> https://github.com/afs/mantis/commit/38dda54
>
> next: prepare it - rename packages and put in under jena-db, in the
> build but otherwise unconnected to the rest of Jena.
>
> then more review.
>
> Then merge to master. Until this point it's just kicking around in the
> repo and not in the way of a release. The next release tick is end
> October (ish).
>
> Andy
Re: Contribution of TDB2
Posted by Andy Seaborne <an...@apache.org>.
We've VOTEd, and the software grants are now on-file.
Next steps:
I'll create a branch and move the code over as-is, old package names,
and all.
If it's done as a pull, history will be preserved. The commits are by my
"@seaborne.org" email address, and also
Natan Cox - fixes a typo in the documentation
https://github.com/afs/mantis/commit/6ca0caf
Adam: code cleaning
https://github.com/afs/mantis/commit/38dda54
next: prepare it - rename packages and put in under jena-db, in the
build but otherwise unconnected to the rest of Jena.
then more review.
Then merge to master. Until this point it's just kicking around in the
repo and not in the way of a release. The next release tick is end
October (ish).
Andy
Re: Contribution of TDB2
Posted by "Bruno P. Kinoshita" <br...@yahoo.com.br.INVALID>.
+1
I see no problem as we won't tell users to move to TDB2. There might be some users who would be interested in testing it, and could help us to enhance the code - though I agree things are rarely ever finished :)
CheersBruno
From: Andy Seaborne <an...@apache.org>
To: dev@jena.apache.org
Sent: Tuesday, 5 September 2017 10:27 AM
Subject: Contribution of TDB2
I'd like to offer TDB2 as a contribution to Apache Jena.
It is by no means "finished" (is anything software ever finished?) but
it is sufficiently there to talk about and kick the tyres; I'll send
notes about the technical state of TDB2 separately in a moment.
* It adds project overhead. The build is only a couple of minutes,
including upload time, (a lot of testing is done with in-memory
databases, like TDB1, which makes setup and teardown of tests much
faster). However, more stuff in the codebase is still more to get right
for a release.
* Support: while TDB2 works, it is unproven and rough round the edges
(unclear errors message, stacktraces rather than meaningful messages
etc) and questions/tickets may be quite hard to respond to (complex
situations; no track record of what's gone wrong before). Testing on MS
Windows or macOS has not been done.
* Integration: It is not stable enough to be used in mainstream Fuseki2
as the default persistent database. I see this as happening gradually.
There is a Fuseki+TDB2 jar built at the moment but you have to use
configuration files to setup the database.
https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
* Initial status: I suggest we label it "incoming" or some such - it
would not hold up a release for example, and not everything might be
fixed at the point of a Jena release. It shouldn't stop the main code
set that users rely on getting out in the usual cycle.
* Legal issues: the code has always been under the Apache License
(original work was done as part of grant from the UK gov which required
open source deliverables) but it would be simpler and cleaner for, e.g.
moving or fixing copyright/licence notices around, for there to be
software grants. The two parties involved are myself and Epimorphics
Ltd. Both parties are willing to submit the paperwork.
* A project vote. This is a new set of modules so a formal, explicit
vote seems worthwhile.
Thoughts, concerns, questions?
Andy
The code:
https://github.com/afs/mantis
(I can clean this up before integration as to what we agree on, so that
there is less noise and churn later).
Builds:
org.seaborne.mantis:tdb2
org.seaborne.mantis:fuseki-tdb2-server
Re: Contribution of TDB2
Posted by Andy Seaborne <an...@apache.org>.
May well be, though databases are very "sticky" so it would be a very long
migration timeline. There could be a migration path - "compact" a TDB1
database into a TDB2 area. Such an import-copy would work for any dataset,
and combination of files, making it a data loader tool.
I can imagine a faster compaction specific to TDB2 that is index-by-index,
but does not touch nodes, as an intermediate compaction - something I
haven't tried at all. Applying it index-by-index would break up the length
of time the system is in maintenance. The simple, and easy to get working
reliably, quad-copy-compaction as the first version of the process has a
lot going for it.
Andy
On 8 September 2017 at 19:18, Adam Soroka <aj...@apache.org> wrote:
> Just catching up.
>
> Sounds great, Andy, and thanks! +1
>
> One question-- as our resident expert on both TDB1 and TDB2 (:grin:) are
> you expecting that TDB2 would eventually replace TDB1 fully and we would
> gracefully move TDB1 our of the core in the same way as, for example,
> Fuseki1 or SDB? It sounds to me like there would be no reason not too, as
> TDB2 matures and becomes stable and widely supported.
>
>
> ajs6f
>
> On 2017-09-08 14:13, Andy Seaborne <an...@apache.org> wrote:
> > To add it to Fuseki.war is (= should be) a matter of adding the
> dependency:
> >
> > <dependency>
> > <groupId>org.seaborne.mantis</groupId>
> > <artifactId>tdb2</artifactId>
> > <version>0.3.0</version>
> > </dependency>
> >
> > It does not need to do anything with jena-fuseki-core (that depends on
> > TDB1) nor appear in the UI so no changes needed. System initialization is
> > by Jena's use of ServiceLoader. (I haven't tried - I'm on a chromebook
> > ATM). The standalone server is a copy of the Jena one with the dependency
> > added.
> >
> > I don't what the size change is - 750K-1M.
> >
> > If there is something else happening, do let me know.
> >
> > Andy
> >
> >
> > On 7 September 2017 at 14:43, Chris Tomlinson <
> chris.j.tomlinson@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > We very much would like to see TDB2 incorporated into Jena as an
> option,
> > > especially for the online compaction. I spent a bit of time trying to
> build
> > > a fuseki-tdb2-war module but it will be much easier if TDB2 is
> incorporated
> > > into Jena (we deploy fuseki as war into tomcat rather than the
> > > fuseki-tbd2-server approach).
> > >
> > > Thanks,
> > > Chris
> > >
> > > > On Sep 4, 2017, at 5:27 PM, Andy Seaborne <an...@apache.org> wrote:
> > > >
> > > > I'd like to offer TDB2 as a contribution to Apache Jena.
> > > >
> > > > It is by no means "finished" (is anything software ever finished?)
> but
> > > it is sufficiently there to talk about and kick the tyres; I'll send
> notes
> > > about the technical state of TDB2 separately in a moment.
> > > >
> > > > * It adds project overhead. The build is only a couple of minutes,
> > > including upload time, (a lot of testing is done with in-memory
> databases,
> > > like TDB1, which makes setup and teardown of tests much faster).
> However,
> > > more stuff in the codebase is still more to get right for a release.
> > > >
> > > > * Support: while TDB2 works, it is unproven and rough round the edges
> > > (unclear errors message, stacktraces rather than meaningful messages
> etc)
> > > and questions/tickets may be quite hard to respond to (complex
> situations;
> > > no track record of what's gone wrong before). Testing on MS Windows or
> > > macOS has not been done.
> > > >
> > > > * Integration: It is not stable enough to be used in mainstream
> Fuseki2
> > > as the default persistent database. I see this as happening gradually.
> > > There is a Fuseki+TDB2 jar built at the moment but you have to use
> > > configuration files to setup the database.
> > > >
> > > > https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
> > > >
> > > > * Initial status: I suggest we label it "incoming" or some such - it
> > > > would not hold up a release for example, and not everything might be
> > > fixed at the point of a Jena release. It shouldn't stop the main code
> set
> > > that users rely on getting out in the usual cycle.
> > > >
> > > > * Legal issues: the code has always been under the Apache License
> > > (original work was done as part of grant from the UK gov which required
> > > open source deliverables) but it would be simpler and cleaner for, e.g.
> > > moving or fixing copyright/licence notices around, for there to be
> software
> > > grants. The two parties involved are myself and Epimorphics Ltd. Both
> > > parties are willing to submit the paperwork.
> > > >
> > > > * A project vote. This is a new set of modules so a formal, explicit
> > > vote seems worthwhile.
> > > >
> > > > Thoughts, concerns, questions?
> > > >
> > > > Andy
> > > >
> > > > The code:
> > > > https://github.com/afs/mantis
> > > > (I can clean this up before integration as to what we agree on, so
> that
> > > there is less noise and churn later).
> > > >
> > > > Builds:
> > > > org.seaborne.mantis:tdb2
> > > > org.seaborne.mantis:fuseki-tdb2-server
> > > >
> > >
> > >
> >
>
Re: Contribution of TDB2
Posted by Adam Soroka <aj...@apache.org>.
Just catching up.
Sounds great, Andy, and thanks! +1
One question-- as our resident expert on both TDB1 and TDB2 (:grin:) are you expecting that TDB2 would eventually replace TDB1 fully and we would gracefully move TDB1 our of the core in the same way as, for example, Fuseki1 or SDB? It sounds to me like there would be no reason not too, as TDB2 matures and becomes stable and widely supported.
ajs6f
On 2017-09-08 14:13, Andy Seaborne <an...@apache.org> wrote:
> To add it to Fuseki.war is (= should be) a matter of adding the dependency:
>
> <dependency>
> <groupId>org.seaborne.mantis</groupId>
> <artifactId>tdb2</artifactId>
> <version>0.3.0</version>
> </dependency>
>
> It does not need to do anything with jena-fuseki-core (that depends on
> TDB1) nor appear in the UI so no changes needed. System initialization is
> by Jena's use of ServiceLoader. (I haven't tried - I'm on a chromebook
> ATM). The standalone server is a copy of the Jena one with the dependency
> added.
>
> I don't what the size change is - 750K-1M.
>
> If there is something else happening, do let me know.
>
> Andy
>
>
> On 7 September 2017 at 14:43, Chris Tomlinson <ch...@gmail.com>
> wrote:
>
> > Hi,
> >
> > We very much would like to see TDB2 incorporated into Jena as an option,
> > especially for the online compaction. I spent a bit of time trying to build
> > a fuseki-tdb2-war module but it will be much easier if TDB2 is incorporated
> > into Jena (we deploy fuseki as war into tomcat rather than the
> > fuseki-tbd2-server approach).
> >
> > Thanks,
> > Chris
> >
> > > On Sep 4, 2017, at 5:27 PM, Andy Seaborne <an...@apache.org> wrote:
> > >
> > > I'd like to offer TDB2 as a contribution to Apache Jena.
> > >
> > > It is by no means "finished" (is anything software ever finished?) but
> > it is sufficiently there to talk about and kick the tyres; I'll send notes
> > about the technical state of TDB2 separately in a moment.
> > >
> > > * It adds project overhead. The build is only a couple of minutes,
> > including upload time, (a lot of testing is done with in-memory databases,
> > like TDB1, which makes setup and teardown of tests much faster). However,
> > more stuff in the codebase is still more to get right for a release.
> > >
> > > * Support: while TDB2 works, it is unproven and rough round the edges
> > (unclear errors message, stacktraces rather than meaningful messages etc)
> > and questions/tickets may be quite hard to respond to (complex situations;
> > no track record of what's gone wrong before). Testing on MS Windows or
> > macOS has not been done.
> > >
> > > * Integration: It is not stable enough to be used in mainstream Fuseki2
> > as the default persistent database. I see this as happening gradually.
> > There is a Fuseki+TDB2 jar built at the moment but you have to use
> > configuration files to setup the database.
> > >
> > > https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
> > >
> > > * Initial status: I suggest we label it "incoming" or some such - it
> > > would not hold up a release for example, and not everything might be
> > fixed at the point of a Jena release. It shouldn't stop the main code set
> > that users rely on getting out in the usual cycle.
> > >
> > > * Legal issues: the code has always been under the Apache License
> > (original work was done as part of grant from the UK gov which required
> > open source deliverables) but it would be simpler and cleaner for, e.g.
> > moving or fixing copyright/licence notices around, for there to be software
> > grants. The two parties involved are myself and Epimorphics Ltd. Both
> > parties are willing to submit the paperwork.
> > >
> > > * A project vote. This is a new set of modules so a formal, explicit
> > vote seems worthwhile.
> > >
> > > Thoughts, concerns, questions?
> > >
> > > Andy
> > >
> > > The code:
> > > https://github.com/afs/mantis
> > > (I can clean this up before integration as to what we agree on, so that
> > there is less noise and churn later).
> > >
> > > Builds:
> > > org.seaborne.mantis:tdb2
> > > org.seaborne.mantis:fuseki-tdb2-server
> > >
> >
> >
>
Re: Contribution of TDB2
Posted by Chris Tomlinson <ch...@gmail.com>.
Hi Andy,
Thanks for the tip. I’m somewhat slow with proper maven use.
I cloned mantis and did
mvn install
and then cloned jena, updated the jena-fuseki-war pom.xml with the dependency and did a
mvn install
in jena and
mvn package
in jena-fuseki-war and produced the desired war file with tdb2-0.3.0.jar.
I then updated the configuration file with:
> @prefix tdb2: <http://jena.apache.org/2016/tdb#> .
>
> . . .
>
> [] ja:loadClass "org.seaborne.tdb2.TDB2" .
> tdb2:DatasetTDB2 rdfs:subClassOf ja:RDFDataset .
>
> . . .
>
> :dataset_test rdf:type tdb2:DatasetTDB2 ;
> tdb2:location "/etc/fuseki/databases/test" ;
> tdb2:unionDefaultGraph true ;
And fuseki seems happy. Now for much testing.
Thanks again,
Chris
> On Sep 8, 2017, at 1:13 PM, Andy Seaborne <an...@apache.org> wrote:
>
> To add it to Fuseki.war is (= should be) a matter of adding the dependency:
>
> <dependency>
> <groupId>org.seaborne.mantis</groupId>
> <artifactId>tdb2</artifactId>
> <version>0.3.0</version>
> </dependency>
>
> It does not need to do anything with jena-fuseki-core (that depends on
> TDB1) nor appear in the UI so no changes needed. System initialization is
> by Jena's use of ServiceLoader. (I haven't tried - I'm on a chromebook
> ATM). The standalone server is a copy of the Jena one with the dependency
> added.
>
> I don't what the size change is - 750K-1M.
>
> If there is something else happening, do let me know.
>
> Andy
>
>
> On 7 September 2017 at 14:43, Chris Tomlinson <ch...@gmail.com>
> wrote:
>
>> Hi,
>>
>> We very much would like to see TDB2 incorporated into Jena as an option,
>> especially for the online compaction. I spent a bit of time trying to build
>> a fuseki-tdb2-war module but it will be much easier if TDB2 is incorporated
>> into Jena (we deploy fuseki as war into tomcat rather than the
>> fuseki-tbd2-server approach).
>>
>> Thanks,
>> Chris
>>
>>> On Sep 4, 2017, at 5:27 PM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> I'd like to offer TDB2 as a contribution to Apache Jena.
>>>
>>> It is by no means "finished" (is anything software ever finished?) but
>> it is sufficiently there to talk about and kick the tyres; I'll send notes
>> about the technical state of TDB2 separately in a moment.
>>>
>>> * It adds project overhead. The build is only a couple of minutes,
>> including upload time, (a lot of testing is done with in-memory databases,
>> like TDB1, which makes setup and teardown of tests much faster). However,
>> more stuff in the codebase is still more to get right for a release.
>>>
>>> * Support: while TDB2 works, it is unproven and rough round the edges
>> (unclear errors message, stacktraces rather than meaningful messages etc)
>> and questions/tickets may be quite hard to respond to (complex situations;
>> no track record of what's gone wrong before). Testing on MS Windows or
>> macOS has not been done.
>>>
>>> * Integration: It is not stable enough to be used in mainstream Fuseki2
>> as the default persistent database. I see this as happening gradually.
>> There is a Fuseki+TDB2 jar built at the moment but you have to use
>> configuration files to setup the database.
>>>
>>> https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
>>>
>>> * Initial status: I suggest we label it "incoming" or some such - it
>>> would not hold up a release for example, and not everything might be
>> fixed at the point of a Jena release. It shouldn't stop the main code set
>> that users rely on getting out in the usual cycle.
>>>
>>> * Legal issues: the code has always been under the Apache License
>> (original work was done as part of grant from the UK gov which required
>> open source deliverables) but it would be simpler and cleaner for, e.g.
>> moving or fixing copyright/licence notices around, for there to be software
>> grants. The two parties involved are myself and Epimorphics Ltd. Both
>> parties are willing to submit the paperwork.
>>>
>>> * A project vote. This is a new set of modules so a formal, explicit
>> vote seems worthwhile.
>>>
>>> Thoughts, concerns, questions?
>>>
>>> Andy
>>>
>>> The code:
>>> https://github.com/afs/mantis
>>> (I can clean this up before integration as to what we agree on, so that
>> there is less noise and churn later).
>>>
>>> Builds:
>>> org.seaborne.mantis:tdb2
>>> org.seaborne.mantis:fuseki-tdb2-server
>>>
>>
>>
Re: Contribution of TDB2
Posted by Andy Seaborne <an...@apache.org>.
To add it to Fuseki.war is (= should be) a matter of adding the dependency:
<dependency>
<groupId>org.seaborne.mantis</groupId>
<artifactId>tdb2</artifactId>
<version>0.3.0</version>
</dependency>
It does not need to do anything with jena-fuseki-core (that depends on
TDB1) nor appear in the UI so no changes needed. System initialization is
by Jena's use of ServiceLoader. (I haven't tried - I'm on a chromebook
ATM). The standalone server is a copy of the Jena one with the dependency
added.
I don't what the size change is - 750K-1M.
If there is something else happening, do let me know.
Andy
On 7 September 2017 at 14:43, Chris Tomlinson <ch...@gmail.com>
wrote:
> Hi,
>
> We very much would like to see TDB2 incorporated into Jena as an option,
> especially for the online compaction. I spent a bit of time trying to build
> a fuseki-tdb2-war module but it will be much easier if TDB2 is incorporated
> into Jena (we deploy fuseki as war into tomcat rather than the
> fuseki-tbd2-server approach).
>
> Thanks,
> Chris
>
> > On Sep 4, 2017, at 5:27 PM, Andy Seaborne <an...@apache.org> wrote:
> >
> > I'd like to offer TDB2 as a contribution to Apache Jena.
> >
> > It is by no means "finished" (is anything software ever finished?) but
> it is sufficiently there to talk about and kick the tyres; I'll send notes
> about the technical state of TDB2 separately in a moment.
> >
> > * It adds project overhead. The build is only a couple of minutes,
> including upload time, (a lot of testing is done with in-memory databases,
> like TDB1, which makes setup and teardown of tests much faster). However,
> more stuff in the codebase is still more to get right for a release.
> >
> > * Support: while TDB2 works, it is unproven and rough round the edges
> (unclear errors message, stacktraces rather than meaningful messages etc)
> and questions/tickets may be quite hard to respond to (complex situations;
> no track record of what's gone wrong before). Testing on MS Windows or
> macOS has not been done.
> >
> > * Integration: It is not stable enough to be used in mainstream Fuseki2
> as the default persistent database. I see this as happening gradually.
> There is a Fuseki+TDB2 jar built at the moment but you have to use
> configuration files to setup the database.
> >
> > https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
> >
> > * Initial status: I suggest we label it "incoming" or some such - it
> > would not hold up a release for example, and not everything might be
> fixed at the point of a Jena release. It shouldn't stop the main code set
> that users rely on getting out in the usual cycle.
> >
> > * Legal issues: the code has always been under the Apache License
> (original work was done as part of grant from the UK gov which required
> open source deliverables) but it would be simpler and cleaner for, e.g.
> moving or fixing copyright/licence notices around, for there to be software
> grants. The two parties involved are myself and Epimorphics Ltd. Both
> parties are willing to submit the paperwork.
> >
> > * A project vote. This is a new set of modules so a formal, explicit
> vote seems worthwhile.
> >
> > Thoughts, concerns, questions?
> >
> > Andy
> >
> > The code:
> > https://github.com/afs/mantis
> > (I can clean this up before integration as to what we agree on, so that
> there is less noise and churn later).
> >
> > Builds:
> > org.seaborne.mantis:tdb2
> > org.seaborne.mantis:fuseki-tdb2-server
> >
>
>
Re: Contribution of TDB2
Posted by Chris Tomlinson <ch...@gmail.com>.
Hi,
We very much would like to see TDB2 incorporated into Jena as an option, especially for the online compaction. I spent a bit of time trying to build a fuseki-tdb2-war module but it will be much easier if TDB2 is incorporated into Jena (we deploy fuseki as war into tomcat rather than the fuseki-tbd2-server approach).
Thanks,
Chris
> On Sep 4, 2017, at 5:27 PM, Andy Seaborne <an...@apache.org> wrote:
>
> I'd like to offer TDB2 as a contribution to Apache Jena.
>
> It is by no means "finished" (is anything software ever finished?) but it is sufficiently there to talk about and kick the tyres; I'll send notes about the technical state of TDB2 separately in a moment.
>
> * It adds project overhead. The build is only a couple of minutes, including upload time, (a lot of testing is done with in-memory databases, like TDB1, which makes setup and teardown of tests much faster). However, more stuff in the codebase is still more to get right for a release.
>
> * Support: while TDB2 works, it is unproven and rough round the edges (unclear errors message, stacktraces rather than meaningful messages etc) and questions/tickets may be quite hard to respond to (complex situations; no track record of what's gone wrong before). Testing on MS Windows or macOS has not been done.
>
> * Integration: It is not stable enough to be used in mainstream Fuseki2 as the default persistent database. I see this as happening gradually. There is a Fuseki+TDB2 jar built at the moment but you have to use configuration files to setup the database.
>
> https://github.com/afs/mantis/blob/master/use-fuseki-tdb2.md
>
> * Initial status: I suggest we label it "incoming" or some such - it
> would not hold up a release for example, and not everything might be fixed at the point of a Jena release. It shouldn't stop the main code set that users rely on getting out in the usual cycle.
>
> * Legal issues: the code has always been under the Apache License (original work was done as part of grant from the UK gov which required open source deliverables) but it would be simpler and cleaner for, e.g. moving or fixing copyright/licence notices around, for there to be software grants. The two parties involved are myself and Epimorphics Ltd. Both parties are willing to submit the paperwork.
>
> * A project vote. This is a new set of modules so a formal, explicit vote seems worthwhile.
>
> Thoughts, concerns, questions?
>
> Andy
>
> The code:
> https://github.com/afs/mantis
> (I can clean this up before integration as to what we agree on, so that there is less noise and churn later).
>
> Builds:
> org.seaborne.mantis:tdb2
> org.seaborne.mantis:fuseki-tdb2-server
>