You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Mikael Pesonen <mi...@lingsoft.fi> on 2019/10/18 11:55:13 UTC
Automatic data sync between servers
We have 3 servers each running an instance of Jena/Fuseki, one server
being the master. All servers have one graph in common, but they contain
other graphs too which contain server specific stuff and are not to be
synced.
Is there a tool that helps to sync the graph on other servers from
master automatically?
Br,
Mikael
Re: Automatic data sync between servers
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
On 19/10/2019 14:33, Andy Seaborne wrote:
> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
> transactions, it is all related changes).
>
> But if it is one graph out of a dataset, then it is more complicated -
> any update may change both local and replicated graphs together. You
> could have a special dataset that picks one the one graph to
> replicated as a layer over the storage dataset and use Delta for that.
>
> The other approach is if the updates to the replicated graph are
> separate from the other graphs in which case, simply sending the
> changes to each server may be a workable approach for you.
Yes, the updates are done only on the master db, so sending the changes
work in this case. Thanks!
Mikael
>
> Andy
>
> [1] https://afs.github.io/rdf-delta
>
> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>
>> We have 3 servers each running an instance of Jena/Fuseki, one server
>> being the master. All servers have one graph in common, but they
>> contain other graphs too which contain server specific stuff and are
>> not to be synced.
>> Is there a tool that helps to sync the graph on other servers from
>> master automatically?
>>
>> Br,
>> Mikael
>>
>>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND
Re: Automatic data sync between servers
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
On 22/10/2019 16:10, Andy Seaborne wrote:
>
>
> On 21/10/2019 12:19, Mikael Pesonen wrote:
>>
>> While testing RDF Delta, I'm following the instructions and have
>> server running:
>>
>> java -jar delta-server.jar --mem
>> [2019-10-21 13:20:00] Delta INFO Provider: mem
>> [2019-10-21 13:20:00] Config INFO Delta Server port=1066
>> [2019-10-21 13:20:00] Delta INFO RDF Delta 0.8.0
>> 2019-10-12T08:58:34+0000
>> [2019-10-21 13:20:00] Delta INFO No data sources
>>
>> Connecting to it succeeded once
>>
>> ./dcmd ls --server http://localhost:1066/
>> -- No logs --
>>
>> But never since. Restarting server doesn't help and the connection
>> just hangs, no timeout or any error.
>>
>> Any idea what could cause that?
>
> Works for me.
>
> Are you sure the server processed isn't suspended? It does not run in
> the background unless you put it in the background. Just suspending it
> would behave much as you describe.
I'm running the server in separate console. And asked from out tech
support there shouldn't be any firewall messing with intra traffic.
>
> Andy
>
>>
>>
>> Mikael
>>
>> On 19/10/2019 14:33, Andy Seaborne wrote:
>>> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
>>> transactions, it is all related changes).
>>>
>>> But if it is one graph out of a dataset, then it is more complicated
>>> - any update may change both local and replicated graphs together.
>>> You could have a special dataset that picks one the one graph to
>>> replicated as a layer over the storage dataset and use Delta for that.
>>>
>>> The other approach is if the updates to the replicated graph are
>>> separate from the other graphs in which case, simply sending the
>>> changes to each server may be a workable approach for you.
>>>
>>> Andy
>>>
>>> [1] https://afs.github.io/rdf-delta
>>>
>>> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>>>
>>>> We have 3 servers each running an instance of Jena/Fuseki, one
>>>> server being the master. All servers have one graph in common, but
>>>> they contain other graphs too which contain server specific stuff
>>>> and are not to be synced.
>>>> Is there a tool that helps to sync the graph on other servers from
>>>> master automatically?
>>>>
>>>> Br,
>>>> Mikael
>>>>
>>>>
>>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND
Re: Automatic data sync between servers
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
It's working, got
{"value" : "2019-11-11T14:23:23.435+02:00" }
Br
On 11/11/2019 13:20, Andy Seaborne wrote:
>
>
> On 04/11/2019 15:19, Mikael Pesonen wrote:
>>
>> Some further testing: I think server is fine since telnet makes a
>> connection and ngrep says for telnet
>>
>> interface: lo (127.0.0.0/255.0.0.0)
>> filter: (ip or ip6) and ( port 1066 )
>>
>>
>> But client is dead, no trace in ngrep and no error. It just hangs for
>> ./dcmd ls --server http://localhost:1066
>> or
>> ./dcmd ls --server donaldduck
>>
>> --help doesn't mention any way to get more debug info for the client.
>> I guess there isn't any hidden debug switch?
>
> You can put a log4j2.properties in the current directory.
>
> But if it isn't logging anything, it is likely thr equest does not get
> to the server.
>
> Try "curl http://HOST:1066/" which should return a small JSON object
> (can be used for load balancer liveness tests)
>
> Andy
>
>>
>>
>>
>> On 22/10/2019 16:10, Andy Seaborne wrote:
>>>
>>>
>>> On 21/10/2019 12:19, Mikael Pesonen wrote:
>>>>
>>>> While testing RDF Delta, I'm following the instructions and have
>>>> server running:
>>>>
>>>> java -jar delta-server.jar --mem
>>>> [2019-10-21 13:20:00] Delta INFO Provider: mem
>>>> [2019-10-21 13:20:00] Config INFO Delta Server port=1066
>>>> [2019-10-21 13:20:00] Delta INFO RDF Delta 0.8.0
>>>> 2019-10-12T08:58:34+0000
>>>> [2019-10-21 13:20:00] Delta INFO No data sources
>>>>
>>>> Connecting to it succeeded once
>>>>
>>>> ./dcmd ls --server http://localhost:1066/
>>>> -- No logs --
>>>>
>>>> But never since. Restarting server doesn't help and the connection
>>>> just hangs, no timeout or any error.
>>>>
>>>> Any idea what could cause that?
>>>
>>> Works for me.
>>>
>>> Are you sure the server processed isn't suspended? It does not run
>>> in the background unless you put it in the background. Just
>>> suspending it would behave much as you describe.
>>>
>>> Andy
>>>
>>>>
>>>>
>>>> Mikael
>>>>
>>>> On 19/10/2019 14:33, Andy Seaborne wrote:
>>>>> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
>>>>> transactions, it is all related changes).
>>>>>
>>>>> But if it is one graph out of a dataset, then it is more
>>>>> complicated - any update may change both local and replicated
>>>>> graphs together. You could have a special dataset that picks one
>>>>> the one graph to replicated as a layer over the storage dataset
>>>>> and use Delta for that.
>>>>>
>>>>> The other approach is if the updates to the replicated graph are
>>>>> separate from the other graphs in which case, simply sending the
>>>>> changes to each server may be a workable approach for you.
>>>>>
>>>>> Andy
>>>>>
>>>>> [1] https://afs.github.io/rdf-delta
>>>>>
>>>>> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>>>>>
>>>>>> We have 3 servers each running an instance of Jena/Fuseki, one
>>>>>> server being the master. All servers have one graph in common,
>>>>>> but they contain other graphs too which contain server specific
>>>>>> stuff and are not to be synced.
>>>>>> Is there a tool that helps to sync the graph on other servers
>>>>>> from master automatically?
>>>>>>
>>>>>> Br,
>>>>>> Mikael
>>>>>>
>>>>>>
>>>>
>>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND
Re: Automatic data sync between servers
Posted by Andy Seaborne <an...@apache.org>.
On 04/11/2019 15:19, Mikael Pesonen wrote:
>
> Some further testing: I think server is fine since telnet makes a
> connection and ngrep says for telnet
>
> interface: lo (127.0.0.0/255.0.0.0)
> filter: (ip or ip6) and ( port 1066 )
>
>
> But client is dead, no trace in ngrep and no error. It just hangs for
> ./dcmd ls --server http://localhost:1066
> or
> ./dcmd ls --server donaldduck
>
> --help doesn't mention any way to get more debug info for the client. I
> guess there isn't any hidden debug switch?
You can put a log4j2.properties in the current directory.
But if it isn't logging anything, it is likely thr equest does not get
to the server.
Try "curl http://HOST:1066/" which should return a small JSON object
(can be used for load balancer liveness tests)
Andy
>
>
>
> On 22/10/2019 16:10, Andy Seaborne wrote:
>>
>>
>> On 21/10/2019 12:19, Mikael Pesonen wrote:
>>>
>>> While testing RDF Delta, I'm following the instructions and have
>>> server running:
>>>
>>> java -jar delta-server.jar --mem
>>> [2019-10-21 13:20:00] Delta INFO Provider: mem
>>> [2019-10-21 13:20:00] Config INFO Delta Server port=1066
>>> [2019-10-21 13:20:00] Delta INFO RDF Delta 0.8.0
>>> 2019-10-12T08:58:34+0000
>>> [2019-10-21 13:20:00] Delta INFO No data sources
>>>
>>> Connecting to it succeeded once
>>>
>>> ./dcmd ls --server http://localhost:1066/
>>> -- No logs --
>>>
>>> But never since. Restarting server doesn't help and the connection
>>> just hangs, no timeout or any error.
>>>
>>> Any idea what could cause that?
>>
>> Works for me.
>>
>> Are you sure the server processed isn't suspended? It does not run in
>> the background unless you put it in the background. Just suspending it
>> would behave much as you describe.
>>
>> Andy
>>
>>>
>>>
>>> Mikael
>>>
>>> On 19/10/2019 14:33, Andy Seaborne wrote:
>>>> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
>>>> transactions, it is all related changes).
>>>>
>>>> But if it is one graph out of a dataset, then it is more complicated
>>>> - any update may change both local and replicated graphs together.
>>>> You could have a special dataset that picks one the one graph to
>>>> replicated as a layer over the storage dataset and use Delta for that.
>>>>
>>>> The other approach is if the updates to the replicated graph are
>>>> separate from the other graphs in which case, simply sending the
>>>> changes to each server may be a workable approach for you.
>>>>
>>>> Andy
>>>>
>>>> [1] https://afs.github.io/rdf-delta
>>>>
>>>> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>>>>
>>>>> We have 3 servers each running an instance of Jena/Fuseki, one
>>>>> server being the master. All servers have one graph in common, but
>>>>> they contain other graphs too which contain server specific stuff
>>>>> and are not to be synced.
>>>>> Is there a tool that helps to sync the graph on other servers from
>>>>> master automatically?
>>>>>
>>>>> Br,
>>>>> Mikael
>>>>>
>>>>>
>>>
>
Re: Automatic data sync between servers
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
Some further testing: I think server is fine since telnet makes a
connection and ngrep says for telnet
interface: lo (127.0.0.0/255.0.0.0)
filter: (ip or ip6) and ( port 1066 )
But client is dead, no trace in ngrep and no error. It just hangs for
./dcmd ls --server http://localhost:1066
or
./dcmd ls --server donaldduck
--help doesn't mention any way to get more debug info for the client. I
guess there isn't any hidden debug switch?
On 22/10/2019 16:10, Andy Seaborne wrote:
>
>
> On 21/10/2019 12:19, Mikael Pesonen wrote:
>>
>> While testing RDF Delta, I'm following the instructions and have
>> server running:
>>
>> java -jar delta-server.jar --mem
>> [2019-10-21 13:20:00] Delta INFO Provider: mem
>> [2019-10-21 13:20:00] Config INFO Delta Server port=1066
>> [2019-10-21 13:20:00] Delta INFO RDF Delta 0.8.0
>> 2019-10-12T08:58:34+0000
>> [2019-10-21 13:20:00] Delta INFO No data sources
>>
>> Connecting to it succeeded once
>>
>> ./dcmd ls --server http://localhost:1066/
>> -- No logs --
>>
>> But never since. Restarting server doesn't help and the connection
>> just hangs, no timeout or any error.
>>
>> Any idea what could cause that?
>
> Works for me.
>
> Are you sure the server processed isn't suspended? It does not run in
> the background unless you put it in the background. Just suspending it
> would behave much as you describe.
>
> Andy
>
>>
>>
>> Mikael
>>
>> On 19/10/2019 14:33, Andy Seaborne wrote:
>>> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
>>> transactions, it is all related changes).
>>>
>>> But if it is one graph out of a dataset, then it is more complicated
>>> - any update may change both local and replicated graphs together.
>>> You could have a special dataset that picks one the one graph to
>>> replicated as a layer over the storage dataset and use Delta for that.
>>>
>>> The other approach is if the updates to the replicated graph are
>>> separate from the other graphs in which case, simply sending the
>>> changes to each server may be a workable approach for you.
>>>
>>> Andy
>>>
>>> [1] https://afs.github.io/rdf-delta
>>>
>>> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>>>
>>>> We have 3 servers each running an instance of Jena/Fuseki, one
>>>> server being the master. All servers have one graph in common, but
>>>> they contain other graphs too which contain server specific stuff
>>>> and are not to be synced.
>>>> Is there a tool that helps to sync the graph on other servers from
>>>> master automatically?
>>>>
>>>> Br,
>>>> Mikael
>>>>
>>>>
>>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND
Re: Automatic data sync between servers
Posted by Andy Seaborne <an...@apache.org>.
On 21/10/2019 12:19, Mikael Pesonen wrote:
>
> While testing RDF Delta, I'm following the instructions and have server
> running:
>
> java -jar delta-server.jar --mem
> [2019-10-21 13:20:00] Delta INFO Provider: mem
> [2019-10-21 13:20:00] Config INFO Delta Server port=1066
> [2019-10-21 13:20:00] Delta INFO RDF Delta 0.8.0
> 2019-10-12T08:58:34+0000
> [2019-10-21 13:20:00] Delta INFO No data sources
>
> Connecting to it succeeded once
>
> ./dcmd ls --server http://localhost:1066/
> -- No logs --
>
> But never since. Restarting server doesn't help and the connection just
> hangs, no timeout or any error.
>
> Any idea what could cause that?
Works for me.
Are you sure the server processed isn't suspended? It does not run in
the background unless you put it in the background. Just suspending it
would behave much as you describe.
Andy
>
>
> Mikael
>
> On 19/10/2019 14:33, Andy Seaborne wrote:
>> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
>> transactions, it is all related changes).
>>
>> But if it is one graph out of a dataset, then it is more complicated -
>> any update may change both local and replicated graphs together. You
>> could have a special dataset that picks one the one graph to
>> replicated as a layer over the storage dataset and use Delta for that.
>>
>> The other approach is if the updates to the replicated graph are
>> separate from the other graphs in which case, simply sending the
>> changes to each server may be a workable approach for you.
>>
>> Andy
>>
>> [1] https://afs.github.io/rdf-delta
>>
>> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>>
>>> We have 3 servers each running an instance of Jena/Fuseki, one server
>>> being the master. All servers have one graph in common, but they
>>> contain other graphs too which contain server specific stuff and are
>>> not to be synced.
>>> Is there a tool that helps to sync the graph on other servers from
>>> master automatically?
>>>
>>> Br,
>>> Mikael
>>>
>>>
>
Re: Automatic data sync between servers
Posted by Mikael Pesonen <mi...@lingsoft.fi>.
While testing RDF Delta, I'm following the instructions and have server
running:
java -jar delta-server.jar --mem
[2019-10-21 13:20:00] Delta INFO Provider: mem
[2019-10-21 13:20:00] Config INFO Delta Server port=1066
[2019-10-21 13:20:00] Delta INFO RDF Delta 0.8.0
2019-10-12T08:58:34+0000
[2019-10-21 13:20:00] Delta INFO No data sources
Connecting to it succeeded once
./dcmd ls --server http://localhost:1066/
-- No logs --
But never since. Restarting server doesn't help and the connection just
hangs, no timeout or any error.
Any idea what could cause that?
Mikael
On 19/10/2019 14:33, Andy Seaborne wrote:
> RDF Delta [1] does this for a dataset (as "dataset" is the unit of
> transactions, it is all related changes).
>
> But if it is one graph out of a dataset, then it is more complicated -
> any update may change both local and replicated graphs together. You
> could have a special dataset that picks one the one graph to
> replicated as a layer over the storage dataset and use Delta for that.
>
> The other approach is if the updates to the replicated graph are
> separate from the other graphs in which case, simply sending the
> changes to each server may be a workable approach for you.
>
> Andy
>
> [1] https://afs.github.io/rdf-delta
>
> On 18/10/2019 12:55, Mikael Pesonen wrote:
>>
>> We have 3 servers each running an instance of Jena/Fuseki, one server
>> being the master. All servers have one graph in common, but they
>> contain other graphs too which contain server specific stuff and are
>> not to be synced.
>> Is there a tool that helps to sync the graph on other servers from
>> master automatically?
>>
>> Br,
>> Mikael
>>
>>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi
Speech Applications - Language Management - Translation - Reader's and Writer's Tools - Text Tools - E-books and M-books
Mikael Pesonen
System Engineer
e-mail: mikael.pesonen@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND
Re: Automatic data sync between servers
Posted by Andy Seaborne <an...@apache.org>.
RDF Delta [1] does this for a dataset (as "dataset" is the unit of
transactions, it is all related changes).
But if it is one graph out of a dataset, then it is more complicated -
any update may change both local and replicated graphs together. You
could have a special dataset that picks one the one graph to replicated
as a layer over the storage dataset and use Delta for that.
The other approach is if the updates to the replicated graph are
separate from the other graphs in which case, simply sending the changes
to each server may be a workable approach for you.
Andy
[1] https://afs.github.io/rdf-delta
On 18/10/2019 12:55, Mikael Pesonen wrote:
>
> We have 3 servers each running an instance of Jena/Fuseki, one server
> being the master. All servers have one graph in common, but they contain
> other graphs too which contain server specific stuff and are not to be
> synced.
> Is there a tool that helps to sync the graph on other servers from
> master automatically?
>
> Br,
> Mikael
>
>