[LDAP API] encoding refactoring status, 5

[Emmanuel Lécharny <el...@gmail.com>]

Hi !

post-X-mas update...

All the LDAP API tests are passing green after a complete decorator 
removal (except from the DSML code), but a few OSGi ones. Which means 
the basic code is working. I will fix the OSGi tests today  (2 failures, 
5 errors), then check the server.

I'm confident it's going to be done very soon... We are talking about a 
49 000 lines of git diff here, so it's not a small refactoring. From the 
API user PoV, though, it should be pretty transparent.

I had to differ the JNDI code removal, it's heavily used in the server 
tests, and changing that would have taken way to much time. Also it can 
be done in a next iteration.

What will remain to be done as soon as I'm done with both the server and 
the API will be to merge those changes with Stefan's changes. It should 
go smoothly though.


More coming soon !

Re: [LDAP API] encoding refactoring status, 7

[Emmanuel Lécharny <el...@gmail.com>]

Hi !

Almost there... ALl the server-iunteg tests are passing but 4 
replication tests.


I had to fight against two nasty bugs yesterday, and it took me a while 
to understand what was going bad.

The fist big bug was difficult to ffix, because it was difficult to 
isolate. The tests were working in the IDE but everything was blocking 
when ran using 'mvn clean install', so it took a while to point out 
which exact test was blocking. At the end of the day, it was a bad PDU 
that generated an exception (on purpose), with a NoticeOfDisconnect not 
being handled the right way server side: the connection was shutdown too 
early on the server, and the NoD was never sent to the client, which was 
hanging forever.

The second bug was almost 10 years old : when an operation was aborted 
server side for any reason - in that case, we configured the server to 
refuse a request that was bigger than a given size -, a NoD was sent and 
never handled by the client, which simply timed out. That was kind of 
passing because the time out was low (30 seconds), and get unnoticed. 
With the changes I applied in the previous bug, the NoD generated a NPE, 
and then the bug was apparent.

So now, NoD are properly handled on both side (client and server).

Some of the other issues I faced with this refactoring is the fact that 
some of the extended response simply don't have a responseName, which 
makes it complicated to find their factory: I finally opted to stored 
the extendedRequest in the ExtendedFuture data structure, so when we 
receive the un-typed response, I just had to check the request to know 
which kind of response it was.

All of these complications are related to the fact that the API must not 
only support extended ops and controls we know of, but also those we 
have no clue about: we should be able to send an unknown control or 
extended operation, assuming the value is properly encoded and decoded 
by the user.


So when those 4 last failing tests will be fixed, I think we will be 
good to go - most of the checks are done in server-integ -. Not sure we 
will be ok this year, and we will definitively not have a release in 
2018, but that may be ok for the very beginning of 2019 ;-)


Happy new year !

[LDAP API] encoding refactoring status, 6

[Emmanuel Lécharny <el...@gmail.com>]

Hi!


As usual, it always takes way longer than expected to get things done :/


I'm facing a road block atm, in one test: search with transactions 
(where we add 10000 entries using transactions). The problem is that RFC 
5805 specifies that externded responses don't have a responseName (the 
ExtnededResponse OID), which makes it quite tricky to retrieve the 
associated factory (as it's stored in a Map<OID, Factory>).
I implemented a workaround : I temporarily inject the responseName to 
get the extended response properly encoded, then remove it, and on the 
API side, the extendedFuture now contains the extendedRequest, and I 
match the request MessageID with the extendedResponse MessageId, to 
retrieve the responseName, then I can decode the response.

It's quite convoluted...

I'm hitting the road today, so I won't make a lot of progress.