[ChangeLog] Some Q about initialization

[Emmanuel Lecharny <el...@gmail.com>]

Hi,

I was just wondering if we should not be able to initialize the logger
dynamically ? Using an extended operation to start or stop the logger
seems appropriate.

I also think that the Interceptor I'm working on is not really a log
interceptor, as it stores anti-modification, and not the modifications
themselves. Should we create another interceptor, or should we store
the anti-modification somewhere else?

As those anti-modification must be applied in revert order, a file is
not really the most appropriate solution.

For what we want to use it (reverting to a previous state) we can
store the anti-modifications in a stack, in memory.

Thoughts ?

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: [ChangeLog] Some Q about initialization

[Alex Karasulu <ak...@apache.org>]

Hey sorry for getting to this late.

On 10/10/07, Emmanuel Lecharny <el...@gmail.com> wrote:
>
> Hi,
>
> I was just wondering if we should not be able to initialize the logger
> dynamically ? Using an extended operation to start or stop the logger
> seems appropriate.


Sure that sounds like a nice management operation.

I also think that the Interceptor I'm working on is not really a log
> interceptor, as it stores anti-modification, and not the modifications
> themselves. Should we create another interceptor, or should we store
> the anti-modification somewhere else?


We need to track both forward and reverse modifications.  If you look at the
advanced interfaces
carved out it helps us to track what changed when and how.  Answering these
questions gives
us a solid auditing view into the change log.  It also enables patching
entries in both directions
so I'd say it's pretty important to track.

For a simple file based implementation have two files:

forward.ldif
  - append to the tail of the file the forward LDIF operation
reverse.ldif
  - append to the head of the file

This is even easier for an in memory proof of concept implementation that we
would just use for
reverting state in our integration tests.

As those anti-modification must be applied in revert order, a file is
> not really the most appropriate solution.


Can't you append to the head of a file?

For what we want to use it (reverting to a previous state) we can
> store the anti-modifications in a stack, in memory.


Yeah this is fine for an in memory implementation sure.

Thoughts ?
>

Please take a look at some of the interfaces in the change log package.  I
have some interesting ideas
represented but we need to finalize them and get commentary.  After this
mechanism looks like it's working
we can retrofit the implementation there which is just a simple interceptor
at this point to use these interfaces.

Note that there are multiple levels of stores for the ChangeLogStore.  Some
can query things since they are
more sophisticated.  Some just can't do any of that but just log.  I did it
this way so we can have some simple
implementations.  Eventually I see other possibilities.  I really want to
just talk about these things soon.

Alex

Re: [ChangeLog] Some Q about initialization

[Alex Karasulu <ak...@apache.org>]

On 10/10/07, David Jencks <da...@yahoo.com> wrote:
>
> Then the log can also work as a transaction journal.  I have no idea
> how hard figuring out a complete representation of the change is --
> your other post made it sound like it has a lot of complexity.
>

I guess by transaction "journal" you mean transaction log?

Alex

Re: [ChangeLog] Some Q about initialization

[Emmanuel Lecharny <el...@gmail.com>]

On 10/10/07, David Jencks <da...@yahoo.com> wrote:
>
> On Oct 10, 2007, at 5:46 AM, Emmanuel Lecharny wrote:
>
> > Hi,
> >
> > I was just wondering if we should not be able to initialize the logger
> > dynamically ? Using an extended operation to start or stop the logger
> > seems appropriate.
>
> What is the use case for turning it on and off while the server is
> running?

Sometime you want to debug a live server, but you don't want to stop
it and restart it. Not sure that it's a good idea though... (It could
have saved me hours in the past, when I had to wait until 3 am to be
able to stop and restart a production server (WebSphere) just to
activate some logs and to do the reverse operation the night after to
desactivate the logs, and one more day to analyze 4.5 Gb of logs ;)

Usually when you introduce runtime configurability like
> this you need to either synchronize something on the runtime path or
> introduce a volatile to assure that the component is using completely
> initialized and configured objects, and this can easily add up in
> speed costs.

Nothing seems to be free those days ;) Yes, you are right, we will
have to synchronize the flag which set the service. Does it cost a lot
? If we consider that the server is able to deliver around 5400 search
req/s on a laptop, what will be the impact ? How many req/s will we
lose ?

>
> >
> > I also think that the Interceptor I'm working on is not really a log
> > interceptor, as it stores anti-modification, and not the modifications
> > themselves. Should we create another interceptor, or should we store
> > the anti-modification somewhere else?
> >
> > As those anti-modification must be applied in revert order, a file is
> > not really the most appropriate solution.
> >
> > For what we want to use it (reverting to a previous state) we can
> > store the anti-modifications in a stack, in memory.
> >
> > Thoughts ?
>
> Why not store a complete representation of the change as a
> modification and compute the anti-modification only if it is needed?

You can't do that. For a delete operation for instance, you must fetch
the deleted entry before deleting it, because you don't have a copy of
it, and you will lose information. However, it would work for Add and
Modify operations (with a lot of complexity for Modify)

In fact, computing the anti-modification is pretty simple :
- Add ==> Delete using the DN
- Del ==> Fetch the old entry and Add it
- Modify ==> Fetch the previous entry, Delete the modified entry and
Add the fetched entry
- Rename from N1 to N2 ==> Rename from N2 to N1
- Move from N1 to N2 ==> Move from N2 to N1

> Then the log can also work as a transaction journal.  I have no idea
> how hard figuring out a complete representation of the change is --
> your other post made it sound like it has a lot of complexity.

The Add and Delete modification are pretty simple. The modify
operation is damn complex. The rename/move operation potentially
impact a hell lot of entries (can be the whole tree), but is simple
too.

>
> thanks
> david jencks
>
> >
> > --
> > Regards,
> > Cordialement,
> > Emmanuel Lécharny
> > www.iktek.com
>
>


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: [ChangeLog] Some Q about initialization

[David Jencks <da...@yahoo.com>]

On Oct 10, 2007, at 5:46 AM, Emmanuel Lecharny wrote:

> Hi,
>
> I was just wondering if we should not be able to initialize the logger
> dynamically ? Using an extended operation to start or stop the logger
> seems appropriate.

What is the use case for turning it on and off while the server is  
running?  Usually when you introduce runtime configurability like  
this you need to either synchronize something on the runtime path or  
introduce a volatile to assure that the component is using completely  
initialized and configured objects, and this can easily add up in  
speed costs.

>
> I also think that the Interceptor I'm working on is not really a log
> interceptor, as it stores anti-modification, and not the modifications
> themselves. Should we create another interceptor, or should we store
> the anti-modification somewhere else?
>
> As those anti-modification must be applied in revert order, a file is
> not really the most appropriate solution.
>
> For what we want to use it (reverting to a previous state) we can
> store the anti-modifications in a stack, in memory.
>
> Thoughts ?

Why not store a complete representation of the change as a  
modification and compute the anti-modification only if it is needed?   
Then the log can also work as a transaction journal.  I have no idea  
how hard figuring out a complete representation of the change is --  
your other post made it sound like it has a lot of complexity.

thanks
david jencks

>
> -- 
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com