Re: A proposal for a new locking strategy

[Andy Seaborne <an...@apache.org>]

On 26/01/16 21:22, Claude Warren wrote:
> Sorry I have not been participating in this discussion as much as I would
> like, as I have been busy with several projects.
>
> My thought was that the "engine" would work on a single dataset, though
> something that works across datasets in an XA style would also make sense.
> My original idea was something at the graph level reflected in the model to
> support the locking that is found in the current system but allow for
> multiple writers where possible.
>
> This discussion has changed my views somewhat.  I still need to spend more
> time with datasets to understand how they play with models and graphs.
> Part of what drives me to do this is the idea that we could have multiple
> servers able to serve and update data from a "single" dataset/model/graph.

If that is the goal, then the locking part is easier (?) as the 
partitioning is fixed by the physical allocation - it's very like 
ajs6f's graph locking partition.  The partitions could even run 
independently, just with a front-end to direct updates.

The joy then comes in the dataset or the query engine trying to stitch 
the global view back together again.

     Andy

>
> Claude
>
> On Sun, Jan 24, 2016 at 6:01 PM, A. Soroka <aj...@virginia.edu> wrote:
>
>> Claude, can I ask you to write more about `engine` in your outline below?
>> Are you proposing some global entity that manages locks across more than
>> one dataset? Or are you saying that something would be needed to coordinate
>> between graphs, models, and datasets that might have dependencies amongst
>> themselves, because right now, those different types can have independent
>> state (instead of all of them being views on datasets, as in Andy’s idea)?
>> In either case, it seems a little to me like the role of `engine` could be
>> filled by a (as-yet-non-existent) Transaction object, if I am understanding
>> correctly in either case.
>>
>> I definitely would think you would want to acquire locking context from
>> the dataset/model/graph with which you are working (or a view thereon).
>> That would allow for optimizations.
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>>> On Jan 9, 2016, at 8:27 AM, Claude Warren <cl...@xenei.com> wrote:
>>>
>>> structure locks (on indexes) are implementation specific, right?
>>>
>>> I think this means that we need a way to register listeners that act
>>> somewhat like participants in a XA locking session.
>>>
>>> So we have something like
>>>
>>> request for lock received.
>>> lock the triples in the internal engine
>>> check each lock listener and lock the triples in the listener.
>>> if any listener can not lock undo the locks thus far and abort.
>>> otherwise return lock success.
>>>
>>> Perhaps a LockListener interface like:
>>>
>>> boolean exclusiveLock( Triple ... )
>>> void exclusiveUnlock( Triple ... )
>>> boolean sharedLock( Triple ... )
>>> void sharedUnlock( Triple ... )
>>>
>>> where Triple is any valid triple pattern like {<Foo> ANY ANY},  though
>>> perhaps Quads would be a better choice.
>>>
>>> Another possibility is that the engine lock call would return an object
>>> that could then be used as the context for the lock listener.
>>>
>>> LockContext ctxt = engine.lock( Triple ... );
>>> for (LockListener listener : listeners )
>>> {
>>>     listener.exclusiveLock( ctxt, Triple ... );
>>> }
>>>
>>> The above example is missing all the "Back out the locks on failure"
>> code.
>>>
>>> Perhaps the LockContext should be retrieved from the Graph, Model or
>>> Dataset so we have
>>>
>>> LockContext ctxt = dataset.createLockContext()
>>>
>>> engine.WriteLock( ctxt, Triple .... )
>>>
>>> LockContext would have a getListeners() method to provide a set of
>>> listeners so that the engine could call the appropriate listener checks.
>>>
>>>
>>> Thoughts?
>>>
>>> Claude
>>>
>>>
>>> On Fri, Jan 8, 2016 at 9:20 PM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>>> If you want an example to think about ...
>>>>
>>>>    Iterator<Triple> iter = graph.find(S,P,O) ;
>>>>
>>>> How does that work in any locking scheme when there may be writers about
>>>> during iteration.
>>>>
>>>> Look like it needs needs both data locks (on triples) and structure
>> locks
>>>> (on indexes).
>>>>
>>>>         Andy
>>>>
>>>
>>>
>>>
>>> --
>>> I like: Like Like - The likeliest place on the web
>>> <http://like-like.xenei.com>
>>> LinkedIn: http://www.linkedin.com/in/claudewarren
>>
>>
>
>