You are viewing a plain text version of this content. The canonical link for it is here.
Posted to api@directory.apache.org by Thomas Cataldo <tc...@gmail.com> on 2010/03/11 20:17:08 UTC
ldap search problems against openldap
Hi,
Consider the following unit test :
public void testHandcodedSearchOne() throws Exception {
LdapConnection con = new LdapConnection("obm23.buffy.kvm", 389);
con.bind();
Cursor<SearchResponse> result = con.search("dc=local", "(uid=thomas)",
SearchScope.SUBTREE, "*");
assertNotNull(result);
if (!result.next()) {
fail("no entry found while something should match.");
}
result.close();
con.close();
}
This one fails on "no entry found while something should match.". Well ok...
Now consider the second test :
public void testHandcodedSearchAll() throws Exception {
LdapConnection con = new LdapConnection("obm23.buffy.kvm", 389);
con.bind();
Cursor<SearchResponse> result = con.search("dc=local", "(uid=*)",
SearchScope.SUBTREE, "*");
assertNotNull(result);
if (!result.next()) {
fail("no entry found while something should match.");
}
SearchResponse data = result.get();
System.out.println("data: "+data);
result.close();
con.close();
}
This one outputs :
2010-03-11 20:09:15,604 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
data: LdapMessage
message Id : 2
controls :
2010-03-11 20:09:15,618 LdapConnection ERROR - There is no future
asscoiated with operation message ID 2, perhaps the operation would
have been completed
2010-03-11 20:09:15,627 IoHandlerAdapter WARN - EXCEPTION, please
implement org.apache.directory.ldap.client.api.LdapConnection.exceptionCaught()
for proper handling:
org.apache.mina.core.write.WriteToClosedSessionException
at org.apache.mina.core.polling.AbstractPollingIoProcessor.clearWriteRequestQueue(AbstractPollingIoProcessor.java:573)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:525)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.removeSessions(AbstractPollingIoProcessor.java:497)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:61)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:974)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Openldap logs for the second test are :
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 fd=32 ACCEPT from
IP=10.0.0.1:39830 (IP=0.0.0.0:389)
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 op=0 BIND dn="" method=128
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 op=0 RESULT tag=97 err=0 text=
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 op=1 SRCH base="dc=local"
scope=2 deref=3 filter="(uid=*)"
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 op=1 SRCH attr=*
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 op=1 SEARCH RESULT tag=101
err=0 nentries=8 text=
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 op=2 ABANDON msg=2
Mar 11 20:09:14 obm23 slapd[8780]: conn=45 fd=32 closed (connection lost)
So everything seems ok. However, when the filter is "(uid=thomas)"
something seems broken :
Mar 11 20:07:31 obm23 slapd[8780]: conn=42 fd=32 ACCEPT from
IP=10.0.0.1:42442 (IP=0.0.0.0:389)
Mar 11 20:07:31 obm23 slapd[8780]: conn=42 op=0 BIND dn="" method=128
Mar 11 20:07:31 obm23 slapd[8780]: conn=42 op=0 RESULT tag=97 err=0 text=
Mar 11 20:07:31 obm23 slapd[8780]: conn=42 op=1 SRCH base="dc=local"
scope=2 deref=3 filter="(?=undefined)"
Mar 11 20:07:31 obm23 slapd[8780]: conn=42 op=1 SRCH attr=*
Mar 11 20:07:31 obm23 slapd[8780]: conn=42 op=1 SEARCH RESULT tag=101
err=0 nentries=0 text=
Mar 11 20:07:32 obm23 slapd[8780]: conn=42 fd=32 closed (connection lost)
Look at the "(?=undefined)" filter in the logs.
I might be doing something wrong, but I don't see anything obvious. Any help ?
Regards,
Thomas.
Re: ldap search problems against openldap
Posted by Thomas Cataldo <tc...@gmail.com>.
I still have those problems with latest packages.
For the exception I'm seeing, last time I had one of those I solved it with:
public void flushAndClose() {
IoFutureListener<IoFuture> listener = new IoFutureListener<IoFuture>() {
@Override
public void operationComplete(IoFuture future) {
future.getSession().close(true);
}
};
lastFuture.addListener(listener);
}
Re: ldap search problems against openldap
Posted by Thomas Cataldo <tc...@gmail.com>.
On Sun, Mar 14, 2010 at 7:51 AM, Thomas Cataldo <tc...@gmail.com> wrote:
> On Fri, Mar 12, 2010 at 3:33 PM, Kiran Ayyagari <ay...@gmail.com> wrote:
>> hi Thomas,
>>
>> I have fixed the LdapConnection class on trunk, which I think
>> should fix the problems you reported, can you try with the latest
>> trunk?
>
> All my searches now seem to run fine. Good job :-)
>
> I'll make some more tests and tell you if I find anything odd.
>
As my tests where running, I'm now doing some loops around my tests to
check everything is stable
public void testFindLoop() {
LDAPUtils utils = new LDAPUtils("obm23.buffy.kvm", null, null,
"ou=users,dc=zz.com,dc=local");
int count = 1000;
long time = System.currentTimeMillis();
for (int i = 0; i < count; i++) {
LdapConnection con = null;
try {
con = utils.getConnection();
String dn = utils.findUniqueDN("(uid=thomas)", con);
assertNotNull(dn);
} catch (Throwable t) {
t.printStackTrace();
fail(t.getMessage());
} finally {
utils.cleanUp(con);
}
}
time = System.currentTimeMillis() - time;
System.err.println(count + " iterations done in " + time + "ms. "
+ (1000000 / time) + "/sec.");
}
This test shows random stack traces :
2010-03-14 07:59:31,633 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-14 07:59:31,634 DefaultExceptionMonitor WARN - Unexpected exception.
java.nio.channels.ClosedSelectorException
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:66)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at org.apache.mina.transport.socket.nio.NioProcessor.select(NioProcessor.java:69)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:961)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
2010-03-14 07:59:31,639 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-14 07:59:31,644 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-14 07:59:31,650 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
Re: ldap search problems against openldap
Posted by Thomas Cataldo <tc...@gmail.com>.
On Fri, Mar 12, 2010 at 3:33 PM, Kiran Ayyagari <ay...@gmail.com> wrote:
> hi Thomas,
>
> I have fixed the LdapConnection class on trunk, which I think
> should fix the problems you reported, can you try with the latest
> trunk?
All my searches now seem to run fine. Good job :-)
I'll make some more tests and tell you if I find anything odd.
Regards,
Thomas.
Re: ldap search problems against openldap
Posted by Kiran Ayyagari <ay...@gmail.com>.
hi Thomas,
I have fixed the LdapConnection class on trunk, which I think
should fix the problems you reported, can you try with the latest
trunk?
Kiran Ayyagari
Re: ldap search problems against openldap
Posted by Thomas Cataldo <tc...@gmail.com>.
On Thu, Mar 11, 2010 at 9:55 PM, Kiran Ayyagari <ay...@gmail.com> wrote:
> can you paste the code for your getDN() and cleanUp() methods too
private String getDN(Cursor<SearchResponse> cursor) throws Exception {
Entry entry = ((SearchResultEntry) cursor.get()).getEntry();
DN dn = entry.getDn();
return dn.toString();
}
private void cleanUp(Cursor<SearchResponse> cursor) {
if (cursor != null) {
try {
cursor.close();
} catch (Exception e) {
logger.error("Error closing cursor");
}
}
}
Re: ldap search problems against openldap
Posted by Kiran Ayyagari <ay...@gmail.com>.
> public String findUniqueDN(String filter, LdapConnection con)
> throws Exception {
> String dn = null;
>
> Cursor<SearchResponse> cursor = null;
> try {
> cursor = con.search(baseDn, filter, SearchScope.SUBTREE, "*");
> if (cursor.next()) {
> dn = getDN(cursor);
> if (cursor.next()) {
> logger.warn("several entries matched " + filter + " in "
> + baseDn);
> dn = null;
> }
> } else {
> logger.warn("nothing found matching " + filter);
> }
> } catch (Throwable e) {
> logger.error("error searching for " + filter, e);
> } finally {
> cleanUp(cursor);
> }
> return dn;
> }
>
> and a (uid=*) filter I can trigger the following output:
can you paste the code for your getDN() and cleanUp() methods too
Kiran Ayyagari
Re: ldap search problems against openldap
Posted by Thomas Cataldo <tc...@gmail.com>.
On Thu, Mar 11, 2010 at 8:53 PM, Kiran Ayyagari <ay...@gmail.com> wrote:
> hi Thomas,
>
> Thank you for performing the tests to confirm the issue, I have
> created a JIRA issue [1], have fixed it locally and running tests at
> the moment
> Will commit it as soon as the tests pass.
>
> [1] https://issues.apache.org/jira/browse/DIRSERVER-1476
>
Way better, but not there yet.
Using the following method:
public String findUniqueDN(String filter, LdapConnection con)
throws Exception {
String dn = null;
Cursor<SearchResponse> cursor = null;
try {
cursor = con.search(baseDn, filter, SearchScope.SUBTREE, "*");
if (cursor.next()) {
dn = getDN(cursor);
if (cursor.next()) {
logger.warn("several entries matched " + filter + " in "
+ baseDn);
dn = null;
}
} else {
logger.warn("nothing found matching " + filter);
}
} catch (Throwable e) {
logger.error("error searching for " + filter, e);
} finally {
cleanUp(cursor);
}
return dn;
}
and a (uid=*) filter I can trigger the following output:
2010-03-11 21:44:24,000 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 21:44:24,006 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 21:44:24,007 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 21:44:24,016 LDAPUtils WARN - several entries matched
(uid=*) in ou=users,dc=zz.com,dc=local
2010-03-11 21:44:24,018 LdapConnection ERROR - There is no future
asscoiated with operation message ID 2, perhaps the operation would
have been completed
2010-03-11 21:44:24,031 IoHandlerAdapter WARN - EXCEPTION, please
implement org.apache.directory.ldap.client.api.LdapConnection.exceptionCaught()
for proper handling:
java.lang.NullPointerException
at org.apache.directory.ldap.client.api.LdapConnection.peekFromFutureMap(LdapConnection.java:291)
at org.apache.directory.ldap.client.api.LdapConnection.messageReceived(LdapConnection.java:1582)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:713)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
at org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolDecoderOutputImpl.flush(ProtocolCodecFilter.java:375)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:229)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793)
at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:119)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:638)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:598)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:587)
at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:61)
at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:969)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
2010-03-11 21:44:24,033 IoHandlerAdapter WARN - EXCEPTION, please
implement org.apache.directory.ldap.client.api.LdapConnection.exceptionCaught()
for proper handling:
java.lang.NullPointerException
at org.apache.directory.ldap.client.api.LdapConnection.peekFromFutureMap(LdapConnection.java:291)
at org.apache.directory.ldap.client.api.LdapConnection.messa
...
Regards,
Thomas.
Re: ldap search problems against openldap
Posted by Kiran Ayyagari <ay...@gmail.com>.
hi Thomas,
Thank you for performing the tests to confirm the issue, I have
created a JIRA issue [1], have fixed it locally and running tests at
the moment
Will commit it as soon as the tests pass.
[1] https://issues.apache.org/jira/browse/DIRSERVER-1476
Kiran Ayyagari
Re: ldap search problems against openldap
Posted by Thomas Cataldo <tc...@gmail.com>.
On Thu, Mar 11, 2010 at 8:35 PM, Kiran Ayyagari <ay...@gmail.com> wrote:
> hi Thomas,
>
> I think I know where the issue is, one of the FilterTransformer we
> use has been changed will fix ASAP and let you know.
>
> BTW can you do one more test to confirm,
>
> Try for the presence of any other attribute like (objectClass=*), (cn=*) etc
>
For (objectClass=*) :
2010-03-11 20:40:25,363 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 20:40:25,364 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 20:40:25,365 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 20:40:25,368 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 20:40:25,370 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
data: LdapMessage
message Id : 2
controls :
2010-03-11 20:40:25,373 Asn1Decoder WARN - The PDU has been fully
decoded but there are still bytes in the buffer.
2010-03-11 20:40:25,372 LdapConnection ERROR - There is no future
asscoiated with operation message ID 2, perhaps the operation would
have been completed
2010-03-11 20:40:25,388 IoHandlerAdapter WARN - EXCEPTION, please
implement org.apache.directory.ldap.client.api.LdapConnection.exceptionCaught()
for proper handling:
java.lang.NullPointerException
at org.apache.directory.ldap.client.api.LdapConnection.peekFromFutureMap(LdapConnection.java:291)
at org.apache.directory.ldap.client.api.LdapConnection.messageReceived(LdapConnection.java:1582)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailF
> and also try for (objectClass=<any-class-like-person-domain-etc>) , (cn=xx)
(objectClass=posixAccount) => nothing found (while several entries should match)
(cn=Thomas Cataldo) => nothing found (should match too)
>
> th second filter should fail.
>
> Kiran Ayyagari
>
Re: ldap search problems against openldap
Posted by Kiran Ayyagari <ay...@gmail.com>.
hi Thomas,
I think I know where the issue is, one of the FilterTransformer we
use has been changed will fix ASAP and let you know.
BTW can you do one more test to confirm,
Try for the presence of any other attribute like (objectClass=*), (cn=*) etc
and also try for (objectClass=<any-class-like-person-domain-etc>) , (cn=xx)
th second filter should fail.
Kiran Ayyagari