You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Alex Karasulu <ao...@bellsouth.net> on 2006/11/07 04:55:23 UTC
[Mitosis] IM Session with Trustin
Hi all,
I'd like to report on some questions, their answers and some conclusions
reached while Trustin and I were talking on IM about Mitosis:
======================= Start =========================
1). When adding an entry that was already marked entryDeleted true, why
does mitosis perform a modify operation on the entry instead?
Answer: I don't remember maybe I did it this way because of a lack of
knowledge.
2). In the OperationFactory.newAdd(CSN,LdapDN,Attributes) method the
first thing you do is call checkBeforeAdd(LdapDN) whose short body is
printed out below:
private void checkBeforeAdd( LdapDN newEntryName ) throws
NamingException
{
if ( nexus.hasEntry( newEntryName ) )
{
throw new NameAlreadyBoundException(
newEntryName.toString() + " already exists." );
}
}
On add operations where the entry is marked entryDeleted=true, this
method always throws an exception and shorts the progress of the add()
operation. I have modified it to check if the entry is marked as
deleted first before throwing an exception like so:
private void checkBeforeAdd( LdapDN newEntryName ) throws
NamingException
{
if ( nexus.hasEntry( newEntryName ) )
{
Attributes entry = nexus.lookup( newEntryName );
Attribute deleted = entry.get( Constants.ENTRY_DELETED );
Object value = deleted == null ? null : deleted.get();
/*
* Check first if the entry has been marked as deleted before
* throwing an exception and delete the entry if so and return
* without throwing an exception.
*/
if ( value != null && "true".equalsIgnoreCase(
value.toString() ) )
{
return;
}
throw new NameAlreadyBoundException(
newEntryName.toString() + " already exists." );
}
}
Now is this the right thing to do?
Answer: Yes this is good.
3). After making this change I still have some problems. Let me
describe the sequence of events in terms of 2 replicas A and B and an
add operation:
(i) add entry (1) to replica A
Everything is fine up to now. Replication of the add propagates just
fine. I can see the entry in both replica A and B.
(ii) delete entry (1) from replica A
Everything is fine up to now. Replication of the del propagates just
fine. I can not see the entry in either replica A and B. Pulling up
the diagnostic gui I can see that the entry is in the partition but
marked entryDeleted = true.
(iii) add entry (1) back to replica A with same DN and same contents
Here I get a *FAILURE*. The error response I get from the server is:
06:06:49 PM: Failed to add new entry ou=neworganizationalUnit,
dc=example,dc=com Root error: [LDAP: error code 80 - failed to add entry
ou=neworganizationalUnit,dc=example,dc=com: Unexpected exception.]
In fact I get this error if I try to add to either replica A or B. At
this point I started a debug session to determine exactly what is going
on which prevents this 2nd add operation from working.
3a). I noticed that for add you build a CompositeOperation but really
you're only putting into it a single AddEntryOperation. Did you intend
to do this or is it a simple bug?
Answer: Yeah you are right we don't need to wrap it.
3b). To the entry to be added you remove any entryDeleted and entryUUID
attributes and add a entryDelete (false), entryUUID and an entryCSN
attribute. Then execute() is called eventually for the
AddEntryOperation. Here you're checking if the entry is updateable by
calling EntryUtil.isEntryUpdatable. Here's the code for it:
http://svn.apache.org/repos/asf/directory/trunks/apacheds/mitosis/src/main/java/org/apache/directory/mitosis/operation/support/EntryUtil.java
Basically you lookup the entry with a call to the nexus. Then you
return true if the entry does not exist. Is this valid? Meaning if the
entry does not exist in the store how can it be updatable?
You also check if there is no CSN attribute. If there is no attribute
then you return true that the entry is updatable. What does this mean?
It would help if you clarify the meaning of these methods in EntryUtils
with a bit more doco. Also it would be nice if you elaborate more on
how mitosis does not delete entries, and why it does this as well as
when and how it cleans up later. This is all critical stuff for the
architecture that was omitted.
Answer:
(20:52:07) trustin: first, if there's no entryCSN attribute, it means that
(20:52:24) trustin: an entry is added before Mitosis is in action.
(20:52:55) akarasulu: ahh ok
(20:53:02) trustin: so it should be updatable and needs to get the CSN
assigned by modification
(20:53:09) akarasulu: ok
(20:53:23) trustin: and then it compares the CSN
(20:53:37) akarasulu: why do you return true if entry == null ?
(20:54:28) trustin: Semantics of update includes both add and modify..
so it's updatable if there's no such entry.
(20:54:42) akarasulu: ahhh ok so add is considered an update
(20:54:58) trustin: yeah update is all operations that affects the DIT.
3c). The code goes past these checks for this scenario since the entry
does exist and is marked entryDeleted: true and it does have a CSN
attribute. Finally you compare whether this old CSN is older than the
new CSN. If so then you return true.
What is the meaning of this? Is it just an undefined situation where
you should not be changing an entry with CSN that is older than the CSN
of the original entry? Why then return true or false for
isEntryUpdatable? Should you not just throw an exception if this
is just an undefined situtation?
Answer: We need to decide between two CSN's, the existing one or the new
operation at hand.
3d). Next you create glue entries. This is not very clear to me. It
was not in your architecture document. I think it is very important to
define exactly what this is for to understand mitosis.
Inside EntryUtils.createGlueEntries() you are using getSuffix() to walk
a DN. How do you intend to walk the DN? From the top down like so for
example?
ou=system
ou=A,ou=system
ou=B,ou=A,ou=system
For each one you are trying to create a glue entry I think. There was
yet no need for a glue entry but we have to understand what it is and
make sure they don't violate schema or we'll bomb out. Can you
elaborate a little on glue entries?
Answer: will put it into another iteration of the mitosis architecture doco.
3e). In an effort to modify the entry you are removing all attributes
that the two old and new entries do not have in common. However you are
removing these from the cloned version of the old entry. Then you never
use this old entry that has attributes removed from it.
Is this another bug?
Answer: Yes
4) If I restart a server after shuting it down I get the following error
and the server dies:
Caused by: java.net.BindException: Address already in use
We found a way to stop this from happening with the LDAP port used by
the server. Can
we make the mitosis replication service port also behave similarly so
this does not
happen?
Answer: config.setReuseAddress( true );
5). Looks like you have not finished implementing the move operation as
is indicated
here in OperationFactory line 188 in newMove():
if ( !deleteOldRn )
{
throw new OperationNotSupportedException( "deleteOldRn must
be true." );
}
Is it that you did not implement this or that it was not possible
because of some issue?
6). Can you please add some internal documentation to the mitosis code?
Answer: Yep
7). Can you update the archtecture documentation to include information
about glue entries, not deleting and cleaning up stuff?
Answer: Yep
8). Could you also elaborate more on the log and how it is used?
Answer: Yep
9). Can we detect when we are out of sync and email administrators?
(22:46:27) akarasulu: yes right ?
(22:46:33) trustin: yes.
===========================================
Notes:
(1) We need to log when we have dropped operations.
(2) We need to email administrators when we have dropped ops and/or when
we are out
of sync.
Decided it is best to recursively delete entries marked as deleted when
re-adding the
entry from scratch.
Alex
Re: [Mitosis] IM Session with Trustin
Posted by Trustin Lee <tr...@gmail.com>.
On 11/7/06, Trustin Lee <tr...@gmail.com> wrote:
>
>
> 5). Looks like you have not finished implementing the move operation as
> > is indicated
> > here in OperationFactory line 188 in newMove():
> >
> > if ( !deleteOldRn )
> > {
> > throw new OperationNotSupportedException( "deleteOldRn must
> > be true." );
> > }
> >
> > Is it that you did not implement this or that it was not possible
> > because of some issue?
>
>
> If deleteOldRn is false, the move operation is not a move operation but a
> copy operation. This means a newly copied entries have to have different
> UUIDs.
>
I was just trying to write something to remind myself of old memory. Please
ignore this. I can't recall the reason I did yet.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: [Mitosis] IM Session with Trustin
Posted by Trustin Lee <tr...@gmail.com>.
That sentence was a garbage I forgot to clean up. Again, please ignore. ;)
Trustin
On 11/8/06, Alex Karasulu <ao...@bellsouth.net> wrote:
>
> Trustin Lee wrote:
> > On 11/7/06, *Alex Karasulu* <aok123@bellsouth.net
> > <ma...@bellsouth.net>> wrote:
> >
> > 1). When adding an entry that was already marked entryDeleted true,
> why
> > does mitosis perform a modify operation on the entry instead?
> >
> > Answer: I don't remember maybe I did it this way because of a lack
> of
> > knowledge.
> >
> >
> > No, it was to avoid recursive deletion.
> >
> > 5). Looks like you have not finished implementing the move operation
> as
> > is indicated
> > here in OperationFactory line 188 in newMove():
> >
> > if ( !deleteOldRn )
> > {
> > throw new OperationNotSupportedException( "deleteOldRn
> must
> > be true." );
> > }
> >
> > Is it that you did not implement this or that it was not possible
> > because of some issue?
> >
> >
> > If deleteOldRn is false, the move operation is not a move operation but
> > a copy operation.
>
> Hmmm I don't know what this means. Inside ADS the move operation is
> used for when entries are literally moved from one container to another.
>
> So if I have the following entries:
>
> ou=base
> ou=A,ou=base
> ou=B,ou=base
> ou=C,ou=base
>
> A move operation would be needed to place ou=A,ou=base under
> ou=C,ou=base to get the following entry:
>
> ou=A,ou=C,ou=base
>
> This is a move operation. This is why the arguments are oldName,
> newParentName, newRdn, and deleteOldRdn. Now a rename operation does
> not change the parent of the object just it's name, but this propagates
> as move's to children.
>
> This means a newly copied entries have to have
> > different UUIDs.
>
> Does moving an entry require a new UUID?
>
> Hmmmm. Basically renaming of entries is causing an issue and this
> UnsupportedOperationException is being fired. I cannot rename entries
> as long as this is the way it is.
>
> I'm trying to understand why you're pushing a modifyRn into a move
> operation. I need to understand your reasons before I can fix it.
>
> Can you provide some more feedback on how to get rename working and what
> should be the default behavior?
>
> Thanks,
> Alex
>
>
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6
Re: [Mitosis] IM Session with Trustin
Posted by Alex Karasulu <ao...@bellsouth.net>.
Trustin Lee wrote:
> On 11/7/06, *Alex Karasulu* <aok123@bellsouth.net
> <ma...@bellsouth.net>> wrote:
>
> 1). When adding an entry that was already marked entryDeleted true, why
> does mitosis perform a modify operation on the entry instead?
>
> Answer: I don't remember maybe I did it this way because of a lack of
> knowledge.
>
>
> No, it was to avoid recursive deletion.
>
> 5). Looks like you have not finished implementing the move operation as
> is indicated
> here in OperationFactory line 188 in newMove():
>
> if ( !deleteOldRn )
> {
> throw new OperationNotSupportedException( "deleteOldRn must
> be true." );
> }
>
> Is it that you did not implement this or that it was not possible
> because of some issue?
>
>
> If deleteOldRn is false, the move operation is not a move operation but
> a copy operation.
Hmmm I don't know what this means. Inside ADS the move operation is
used for when entries are literally moved from one container to another.
So if I have the following entries:
ou=base
ou=A,ou=base
ou=B,ou=base
ou=C,ou=base
A move operation would be needed to place ou=A,ou=base under
ou=C,ou=base to get the following entry:
ou=A,ou=C,ou=base
This is a move operation. This is why the arguments are oldName,
newParentName, newRdn, and deleteOldRdn. Now a rename operation does
not change the parent of the object just it's name, but this propagates
as move's to children.
This means a newly copied entries have to have
> different UUIDs.
Does moving an entry require a new UUID?
Hmmmm. Basically renaming of entries is causing an issue and this
UnsupportedOperationException is being fired. I cannot rename entries
as long as this is the way it is.
I'm trying to understand why you're pushing a modifyRn into a move
operation. I need to understand your reasons before I can fix it.
Can you provide some more feedback on how to get rename working and what
should be the default behavior?
Thanks,
Alex
Re: [Mitosis] IM Session with Trustin
Posted by Trustin Lee <tr...@gmail.com>.
On 11/7/06, Alex Karasulu <ao...@bellsouth.net> wrote:
>
> 1). When adding an entry that was already marked entryDeleted true, why
> does mitosis perform a modify operation on the entry instead?
>
> Answer: I don't remember maybe I did it this way because of a lack of
> knowledge.
No, it was to avoid recursive deletion.
5). Looks like you have not finished implementing the move operation as
> is indicated
> here in OperationFactory line 188 in newMove():
>
> if ( !deleteOldRn )
> {
> throw new OperationNotSupportedException( "deleteOldRn must
> be true." );
> }
>
> Is it that you did not implement this or that it was not possible
> because of some issue?
If deleteOldRn is false, the move operation is not a move operation but a
copy operation. This means a newly copied entries have to have different
UUIDs.
8). Could you also elaborate more on the log and how it is used?
>
> Answer: Yep
The answer was that logs are used to send the changes made in a replica.
The logs are selected and sent to the other replica when UV differences are
detected. We also calculate PV and UV from the logs. It's implemented with
Derby, an embedded RDBMS, so I used SELECT MAX(...) and MIN(...) to get UV
and PV. I think this is already described in the existing documentation.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41 4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4 455E 1C62 A7DC 0255 ECA6