You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Ca...@ibs-ag.com on 2012/12/07 16:57:49 UTC

Using ApacheDS api SearchCursor , getting Asn1Decoder warning to logs / console..

Hi,   We've starting moving some of our JNDI code to the ApacheDS API. The method below is used to get an Array of entries.
When we close the cursor ( I think), we see the following log message.

[NioProcessor-4] WARN org.apache.directory.shared.asn1.ber.Asn1Decoder - ERR_00043_REMAINING_BYTES_FOR_DECODED_PDU The PDU has been fully decoded but there are still bytes in the buffer.

We get the results we want but is there something else we need to do with the cursor to avoid the message?  Thanks.


       protected Entry[] getEntries(SearchRequest req, long limit) throws LdapException{
              List<Entry> lstEntries = new ArrayList<Entry>();
                     req.setSizeLimit(limit);
                     SearchCursor cursor = connection.search(req);
                     try {
                           while(cursor.next()){
                                  Response response = cursor.get();
                                  Entry entry = ((SearchResultEntry)response).getEntry();
                                  lstEntries.add(entry);
                           }
                           SearchResultDone done = cursor.getSearchResultDone();
                           if(ResultCodeEnum.SUCCESS.equals(done.getLdapResult().getResultCode())){
                                  cursor.close();
                           }
                           else
                           {
                                  log.error(ERROR_LDAP_RESULT, done.getLdapResult().getResultCode().toString());
                           }
                     } catch (Exception e) {
                           e.printStackTrace();
                           try {
                                  cursor.close(e);
                           } catch (Exception e1) {
                                  e1.printStackTrace();
                           }
                     }
              return lstEntries.toArray(new Entry[lstEntries.size()]);
       }

We set the SearchRequest up like this.

SearchRequest req = new SearchRequestImpl();
              try {
                     req.setBase(new Dn(strDefaultDirRootSuffix));
                     req.setScope(SearchScope.SUBTREE);
                     req.setFilter(strSearch);

                     if (attrs != null && attrs.length > 0){
                           req.addAttributes(attrs);
                     } else {
                           req.addAttributes(DEFAULT_ATTRIBUTES);
                     }
              } catch (LdapInvalidDnException e1) {
                     e1.printStackTrace();
              } catch (LdapException e) {
                     e.printStackTrace();
              }


RE: Using ApacheDS api SearchCursor , getting Asn1Decoder warning to logs / console..

Posted by Ca...@ibs-ag.com.
Ok thanks for the detailed clarification! 


-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com] 
Sent: Friday, December 07, 2012 11:22 AM
To: users@directory.apache.org
Subject: Re: Using ApacheDS api SearchCursor , getting Asn1Decoder warning to logs / console..

Le 12/7/12 4:57 PM, Carlo.Accorsi@ibs-ag.com a écrit :
> Hi,   We've starting moving some of our JNDI code to the ApacheDS API. The method below is used to get an Array of entries.
> When we close the cursor ( I think), we see the following log message.
>
> [NioProcessor-4] WARN org.apache.directory.shared.asn1.ber.Asn1Decoder - ERR_00043_REMAINING_BYTES_FOR_DECODED_PDU The PDU has been fully decoded but there are still bytes in the buffer.
>
> We get the results we want but is there something else we need to do with the cursor to avoid the message?  Thanks.
This is a wrong message. It should be at level INFO at best.

What happens is that when the client receives some data from the server (whatever server it can be), it will decode those data and produce LDAP messages. So far, so good. The thing is that we may have received more than one LDAP message in a simple socket read, and the decoder detect that. Here are the two methods that handle the decoding :

1) the main loop, decoding LDAP message :

        List<Message> decodedMessages = new ArrayList<Message>();
        ByteBuffer buf = in.buf();

        decode( buf, messageContainer, decodedMessages );

        for ( Message message : decodedMessages )
        {
            out.write( message );
        }

It calls the decode() message with a ByteBuffer that might contain more than one message, and accumulate the messages in a list that will be processed later :

    private void decode( ByteBuffer buffer, LdapMessageContainer<MessageDecorator<? extends Message>> messageContainer,
        List<Message> decodedMessages ) throws DecoderException
    {
        while ( buffer.hasRemaining() )
        {
            try
            {
                asn1Decoder.decode( buffer, messageContainer );

                if ( messageContainer.getState() == TLVStateEnum.PDU_DECODED )
                {

                    Message message = messageContainer.getMessage();

                    decodedMessages.add( message );

                    messageContainer.clean();
                }
            }
        }
    }

The Asn1Decoder process one single message, and when done, if there is still some data in the buffer, generates some Warning :

    public void decode( ByteBuffer stream, Asn1Container container ) throws DecoderException
    {
        boolean hasRemaining = stream.hasRemaining();
        ...

        while ( hasRemaining )
        {
                ...
                case PDU_DECODED:
                    // We have to deal with the case where there are
                    // more bytes in the buffer, but the PDU has been decoded.
                    LOG.warn( I18n.err(
I18n.ERR_00043_REMAINING_BYTES_FOR_DECODED_PDU ) );

                    hasRemaining = false;

                    break;
            }
        }


Bottom line, I think I will move the Warning to a Debug.

--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com 


Re: Using ApacheDS api SearchCursor , getting Asn1Decoder warning to logs / console..

Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 12/7/12 4:57 PM, Carlo.Accorsi@ibs-ag.com a écrit :
> Hi,   We've starting moving some of our JNDI code to the ApacheDS API. The method below is used to get an Array of entries.
> When we close the cursor ( I think), we see the following log message.
>
> [NioProcessor-4] WARN org.apache.directory.shared.asn1.ber.Asn1Decoder - ERR_00043_REMAINING_BYTES_FOR_DECODED_PDU The PDU has been fully decoded but there are still bytes in the buffer.
>
> We get the results we want but is there something else we need to do with the cursor to avoid the message?  Thanks.
This is a wrong message. It should be at level INFO at best.

What happens is that when the client receives some data from the server
(whatever server it can be), it will decode those data and produce LDAP
messages. So far, so good. The thing is that we may have received more
than one LDAP message in a simple socket read, and the decoder detect
that. Here are the two methods that handle the decoding :

1) the main loop, decoding LDAP message :

        List<Message> decodedMessages = new ArrayList<Message>();
        ByteBuffer buf = in.buf();

        decode( buf, messageContainer, decodedMessages );

        for ( Message message : decodedMessages )
        {
            out.write( message );
        }

It calls the decode() message with a ByteBuffer that might contain more
than one message, and accumulate the messages in a list that will be
processed later :

    private void decode( ByteBuffer buffer,
LdapMessageContainer<MessageDecorator<? extends Message>> messageContainer,
        List<Message> decodedMessages ) throws DecoderException
    {
        while ( buffer.hasRemaining() )
        {
            try
            {
                asn1Decoder.decode( buffer, messageContainer );

                if ( messageContainer.getState() ==
TLVStateEnum.PDU_DECODED )
                {

                    Message message = messageContainer.getMessage();

                    decodedMessages.add( message );

                    messageContainer.clean();
                }
            }
        }
    }

The Asn1Decoder process one single message, and when done, if there is
still some data in the buffer, generates some Warning :

    public void decode( ByteBuffer stream, Asn1Container container )
throws DecoderException
    {
        boolean hasRemaining = stream.hasRemaining();
        ...

        while ( hasRemaining )
        {
                ...
                case PDU_DECODED:
                    // We have to deal with the case where there are
                    // more bytes in the buffer, but the PDU has been
decoded.
                    LOG.warn( I18n.err(
I18n.ERR_00043_REMAINING_BYTES_FOR_DECODED_PDU ) );

                    hasRemaining = false;

                    break;
            }
        }


Bottom line, I think I will move the Warning to a Debug.

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com