Re: svn commit: r1234853 - in /incubator/jena/Jena2/Fuseki/trunk: src-dev/dev/ src/main/java/org/apache/jena/fuseki/ src/main/java/org/apache/jena/fuseki/mgt/ src/main/java/org/apache/jena/fuseki/server/

[Paolo Castagna <ca...@googlemail.com>]

andy@apache.org wrote:
> Framework for management functions (backup, stop server, others to be decided)

Thank you Andy, I've just look at it (not tried yet).
But, it seems to me quite an useful thing (for production systems).

While ago, when I was thinking how a very simple replication scheme (using rsync)
would work, I was planning to have a management function to set Fuseki in read-only
mode (i.e. it sync changes on disk and it stops listening to updates) and then set
it back to read-write mode (if that was the starting configuration). This way people
are sure that all changes have been flushed on disk, no more changes are possible,
therefore it is (right?) possible to copy, backup, manipulate the TDB indexes using
filesystem operations/tools (including rsync).

Does a management function to switch between read-only and read-write mode make
sense to you?

Paolo

Re: svn commit: r1234853 - in /incubator/jena/Jena2/Fuseki/trunk: src-dev/dev/ src/main/java/org/apache/jena/fuseki/ src/main/java/org/apache/jena/fuseki/mgt/ src/main/java/org/apache/jena/fuseki/server/

[Andy Seaborne <an...@apache.org>]

On 24/01/12 08:36, Paolo Castagna wrote:
> andy@apache.org wrote:
>> Framework for management functions (backup, stop server, others to be decided)
>
> Thank you Andy, I've just look at it (not tried yet).
> But, it seems to me quite an useful thing (for production systems).
>
> While ago, when I was thinking how a very simple replication scheme (using rsync)
> would work, I was planning to have a management function to set Fuseki in read-only
> mode (i.e. it sync changes on disk and it stops listening to updates) and then set
> it back to read-write mode (if that was the starting configuration). This way people
> are sure that all changes have been flushed on disk, no more changes are possible,
> therefore it is (right?) possible to copy, backup, manipulate the TDB indexes using
> filesystem operations/tools (including rsync).
>
> Does a management function to switch between read-only and read-write mode make
> sense to you?
>
> Paolo

A feature to put the server (or, rather, a dataset) into read-only mode 
would of itself be useful.

To copy, or rsync, the dataset you need to
(1) make it go read only
(2) wait for write transactions to end
(3) ensure the journal has been written back to the DB

then the DB on disk is consistent and can be copied.

It's not possible to manipulate the indexes while the system is running 
currently so you can't rsync in changes.  Cached information would need 
to (atomically) be dropped and then restart from disk.

A way to have two copies of the DB, one active, one not, and switch 
between them would be a way.

Another way is to put a load balancer in front of two copies and 
start/stop the copies.

I have (this week) used both HAProxy and AWS Load Balancer to do exactly 
that.

	Andy