Re: [Apache Bloodhound] BloodhoundSearch modified

[Olemis Lang <ol...@gmail.com>]

On 12/19/12, Apache Bloodhound <bl...@incubator.apache.org> wrote:
> Page "BloodhoundSearch" was changed by andrej
> Diff URL:
> <https://issues.apache.org/bloodhound/wiki/BloodhoundSearch?action=diff&version=5>
> Revision 5
> Changes:
> -------8<------8<------8<------8<------8<------8<------8<------8<--------
> Index: BloodhoundSearch
> =========================================================================
> --- BloodhoundSearch (version: 4)
> +++ BloodhoundSearch (version: 5)
> @@ -62,4 +62,6 @@
>   * Improve index consistency.
>     * Current implementation uses ITicketChangeListener interface that
> triggers after DB committed. There is some chances to get inconsistency
> between DB and index. In this case reindex is required. One of the possible
> solution is contact Trac community to introduce a new within transaction
> I<Resource>ChangingListener interface, save changes into additional table
> and index changes in separate thread.
>     * Some ticket changes are not reflected in event interfaces e.g. Version
> and Component renaming. One of the possible solution is to contact Trac
> community to introduce new interfaces.
> +   * Alternative solution is triggering events from SQL proxy developed on
> multi-product branch
> +   * Usage of DB triggers is another alternative

side note : for some reason Trac-dev are not very fond of triggers ...
maybe because each DB backend provider implements triggers in a
different manner ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

Re: [Apache Bloodhound] BloodhoundSearch modified

[Branko Čibej <br...@wandisco.com>]

On 19.12.2012 18:03, Olemis Lang wrote:
> On 12/19/12, Apache Bloodhound <bl...@incubator.apache.org> wrote:
>> Page "BloodhoundSearch" was changed by andrej
>> Diff URL:
>> <https://issues.apache.org/bloodhound/wiki/BloodhoundSearch?action=diff&version=5>
>> Revision 5
>> Changes:
>> -------8<------8<------8<------8<------8<------8<------8<------8<--------
>> Index: BloodhoundSearch
>> =========================================================================
>> --- BloodhoundSearch (version: 4)
>> +++ BloodhoundSearch (version: 5)
>> @@ -62,4 +62,6 @@
>>   * Improve index consistency.
>>     * Current implementation uses ITicketChangeListener interface that
>> triggers after DB committed. There is some chances to get inconsistency
>> between DB and index. In this case reindex is required. One of the possible
>> solution is contact Trac community to introduce a new within transaction
>> I<Resource>ChangingListener interface, save changes into additional table
>> and index changes in separate thread.
>>     * Some ticket changes are not reflected in event interfaces e.g. Version
>> and Component renaming. One of the possible solution is to contact Trac
>> community to introduce new interfaces.
>> +   * Alternative solution is triggering events from SQL proxy developed on
>> multi-product branch
>> +   * Usage of DB triggers is another alternative
> side note : for some reason Trac-dev are not very fond of triggers ...
> maybe because each DB backend provider implements triggers in a
> different manner ?

At least SQLite had bugs in certain cases of trigger usage until quite
recently. That could be one reason.

The more likely reason is that Trac doesn't actually use a relational
database. It uses a persistent data store that depends on some minor
side effects of the feature set of a relational database. :)

-- Brane

-- 
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com

Re: [Apache Bloodhound] BloodhoundSearch modified

[Andrej Golcov <an...@digiverse.si>]

Thanks to Olemis to bring this subject into discussion.

> +   * Usage of DB triggers is another alternative
> side note : for some reason Trac-dev are not very fond of triggers ...
> maybe because each DB backend provider implements triggers in a
> different manner
I'm also not a big fan od DB level hooks but we have to take all
possibilities in consideration :)

Talking about index/DB consistency, we have a transactional DB and
non-transactional index. And we need some mechanism to have index as
much as possible consistent to DB in order to avoid often full
re-indexes.

Talking about ticket indexing, I see two problems so far:
 - Version, Component and Product updates are not reflected in events
that can be hooked for index update
 - ITicketChangeListener events are triggered after DB transaction
commit that can potentially cause index inconsistency in case of index
backend is unavailable or process terminated unexpectedly.

IMHO, the most clean solution is adding support for this in Trac core.
The suggestion is to introduce new interfaces such as
ITicketChangingListener, IVersionChangingListener,
IComponentChangingListener etc. that are triggered within
transactional scope. These interfaces are aimed for short running
operations within transaction but not for long running tasks such as
mail sending etc.

The search plugin will hook on these events. When called within
transaction (for example by ITicketChangingListener) the search plugin
can insert a record into new table e.g. index_tasks with changed
ticket id, action name (changed/new/deleted) etc.

Another out-of-transaction task can SELECT FROM index_tasks and
perform real indexing. The task can be executed by cron job or/and
within the same request that changed the ticket.

Uf, a bit complicated, isn't it?
What do you think, does it make sense? Other possible solutions are
collected here:
https://issues.apache.org/bloodhound/wiki/BloodhoundSearch#consistency

What do you think if we contact Trac community with suggestion (and
code) of introducing new interfaces like ITicketChangingListener and
others?

Regards, Andrej