Tracking deletions

[Jeff Faath <jf...@apache.org>]

Anyone have opinions on tracking deletions?  As part of the subscription
API, deletions of subscribed entities must be returned in the results.  Not
to mention, per JUDDI-213, "virtually" deleted entities must be returned.
These are defined as entities that at one point matched the subscription
filter criteria, but due to an update, no longer match.

I have a couple ideas, ranging from simple to complicated.

Below is a summary of my ideas.

1) Use operationalInfo:  

The operational info structure houses all entity keys (and keeps track of
their created and modification dates).  When an entity is deleted, we could
leave the record in this table.  That way, we'd have an echo of deleted
records.  I don't like this method because checking for deleted records is a
pain (you'd have to left join against each entity table to determine keys in
one table and not another) and it doesn't handle virtually deleted entities.

2) Persist initial subscription matches (my favored approach):

- When a subscription is first saved, the filter critiera is used to
determine all the keys that match.

- Matching keys are persisted along with the subscription.

- At a later point, when subscription results are requested, we can compare
the new results with the initial results to determine the deleted entities.
This will also handle "virtually" deleted entities.

3) Keep a change record journal:

If/when we decide to implement replication, one of the requirements is to
maintain a record of the changes to entities in the registry.  It seems like
this could be used to determine deleted entities.  However, I don't have an
idea as to the logisitics of this method.  Not to mention, determining how
to track changes is a project in itself (Kurt, have you read chapter 7
yet???).

Option 2 is the best approach in my opinion.  It should be relatively easy
to implement.  Option 3 may be the best long-term approach, but like I said,
it's complicated and I don't have a clear idea of how to use it with the
subscription requirements.

Any opinions/suggestions?

-JF

Re: Tracking deletions

[Tom Cunningham <tc...@redhat.com>]

I vote 2.        The only thing I think we need to be concerned about is 
that I think you can choose your own keys in v3 (maybe I'm confused, but 
I thought I read that in the spec).      If that's so, we'd need to 
update the subscription set matches after a key update.

Jeff Faath wrote:
> Anyone have opinions on tracking deletions?  As part of the subscription
> API, deletions of subscribed entities must be returned in the results.  Not
> to mention, per JUDDI-213, "virtually" deleted entities must be returned.
> These are defined as entities that at one point matched the subscription
> filter criteria, but due to an update, no longer match.
>
> I have a couple ideas, ranging from simple to complicated.
>
> Below is a summary of my ideas.
>
> 1) Use operationalInfo:  
>
> The operational info structure houses all entity keys (and keeps track of
> their created and modification dates).  When an entity is deleted, we could
> leave the record in this table.  That way, we'd have an echo of deleted
> records.  I don't like this method because checking for deleted records is a
> pain (you'd have to left join against each entity table to determine keys in
> one table and not another) and it doesn't handle virtually deleted entities.
>
> 2) Persist initial subscription matches (my favored approach):
>
> - When a subscription is first saved, the filter critiera is used to
> determine all the keys that match.
>
> - Matching keys are persisted along with the subscription.
>
> - At a later point, when subscription results are requested, we can compare
> the new results with the initial results to determine the deleted entities.
> This will also handle "virtually" deleted entities.
>
> 3) Keep a change record journal:
>
> If/when we decide to implement replication, one of the requirements is to
> maintain a record of the changes to entities in the registry.  It seems like
> this could be used to determine deleted entities.  However, I don't have an
> idea as to the logisitics of this method.  Not to mention, determining how
> to track changes is a project in itself (Kurt, have you read chapter 7
> yet???).
>
> Option 2 is the best approach in my opinion.  It should be relatively easy
> to implement.  Option 3 may be the best long-term approach, but like I said,
> it's complicated and I don't have a clear idea of how to use it with the
> subscription requirements.
>
> Any opinions/suggestions?
>
> -JF
>
>