You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by "Xu, Yaning" <ya...@intel.com> on 2015/07/02 09:39:08 UTC
How to implement getIdentities(int start, int limit) more
effectively
Hi all,
As we have discussed https://issues.apache.org/jira/browse/DIRKRB-295 , IdentityService.getIdentities() should be removed. However, to implement getIdentities(int start, int limit), which returns a sorted result, I have to get all identities from the ldap server and then sort them. Is it possible for the Ldap server to return a sorted result for the client connection?
In [1], I see a test about Page Search on client side testPagedSearchWrongCookie(), and it uses pagedSearchControl.setSize(3) to set the page size, the problem are:
*how can I get a search result based on the start index and pageSize, like I may use getIdentities(int start, int limit) to return a result;
*how can I get a sorted result from the server, so that I don't have to get all the entries and then sort them to get the result;
[1] https://svn.apache.org/repos/asf/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PagedSearchIT.java
public List<String> getIdentities(int start, int limit) {
List<String> identities = getIdentities();
if (limit == -1) {
return identities;
}
return getIdentities().subList(start, start + limit);
}
private List<String> getIdentities() {
List<String> identityNames = new ArrayList<>();
EntryCursor cursor;
Entry entry;
try {
cursor = connection.search( getConfig().getString("base_dn"),
"(objectclass=*)", SearchScope.ONELEVEL, KerberosAttribute.KRB5_PRINCIPAL_NAME_AT);
if (cursor == null) {
return null;
}
while (cursor.next()) {
entry = cursor.get();
identityNames.add(entry.get(KerberosAttribute.KRB5_PRINCIPAL_NAME_AT).getString());
}
cursor.close();
Collections.sort(identityNames);
} catch (LdapException e) {
e.printStackTrace();
} catch (CursorException e) {
e.printStackTrace();
}
return identityNames;
}
Regards,
Yaning
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Xu, Yaning" <ya...@intel.com>.
Sorry I forgot to mention my many thanks to Stefan :). Further pointers are much appreciated.
Yaning
From: Xu, Yaning [mailto:yaning.xu@intel.com]
Sent: Thursday, July 02, 2015 3:39 PM
To: Apache Directory Developers List
Subject: How to implement getIdentities(int start, int limit) more effectively
Hi all,
As we have discussed https://issues.apache.org/jira/browse/DIRKRB-295 , IdentityService.getIdentities() should be removed. However, to implement getIdentities(int start, int limit), which returns a sorted result, I have to get all identities from the ldap server and then sort them. Is it possible for the Ldap server to return a sorted result for the client connection?
In [1], I see a test about Page Search on client side testPagedSearchWrongCookie(), and it uses pagedSearchControl.setSize(3) to set the page size, the problem are:
*how can I get a search result based on the start index and pageSize, like I may use getIdentities(int start, int limit) to return a result;
*how can I get a sorted result from the server, so that I don't have to get all the entries and then sort them to get the result;
[1] https://svn.apache.org/repos/asf/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PagedSearchIT.java
public List<String> getIdentities(int start, int limit) {
List<String> identities = getIdentities();
if (limit == -1) {
return identities;
}
return getIdentities().subList(start, start + limit);
}
private List<String> getIdentities() {
List<String> identityNames = new ArrayList<>();
EntryCursor cursor;
Entry entry;
try {
cursor = connection.search( getConfig().getString("base_dn"),
"(objectclass=*)", SearchScope.ONELEVEL, KerberosAttribute.KRB5_PRINCIPAL_NAME_AT);
if (cursor == null) {
return null;
}
while (cursor.next()) {
entry = cursor.get();
identityNames.add(entry.get(KerberosAttribute.KRB5_PRINCIPAL_NAME_AT).getString());
}
cursor.close();
Collections.sort(identityNames);
} catch (LdapException e) {
e.printStackTrace();
} catch (CursorException e) {
e.printStackTrace();
}
return identityNames;
}
Regards,
Yaning
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
Ah, this sounds perfect indeed to go!! As Emmanuel mentioned, "Cursor already implements Iterable".
-----Original Message-----
From: Stefan Seelmann [mailto:mail@stefan-seelmann.de]
Sent: Monday, July 06, 2015 2:42 PM
To: kerby@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
On 07/06/2015 08:33 AM, Xu, Yaning wrote:
> Thanks for Kai and Emmanuel's good discussion.
>
>>> ... keep it simple and use List<String> getIdentiies() at the moment.
>
> This sounds good to me to proceed. Thanks!
Just a suggestion: Use java.lang.Iterable<String> (not Iterator) as return type. For the first implementation you can still return a List<String> as java.util.List implements this interface. But you don't expose any implementation detail. Later you can change the actual return type to something more sophisticated and even extend the return type without breaking the interface.
Kind Regards,
Stefan
Re: How to implement getIdentities(int start, int limit) more
effectively
Posted by Stefan Seelmann <ma...@stefan-seelmann.de>.
On 07/06/2015 08:33 AM, Xu, Yaning wrote:
> Thanks for Kai and Emmanuel's good discussion.
>
>>> ... keep it simple and use List<String> getIdentiies() at the moment.
>
> This sounds good to me to proceed. Thanks!
Just a suggestion: Use java.lang.Iterable<String> (not Iterator) as
return type. For the first implementation you can still return a
List<String> as java.util.List implements this interface. But you don't
expose any implementation detail. Later you can change the actual return
type to something more sophisticated and even extend the return type
without breaking the interface.
Kind Regards,
Stefan
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Xu, Yaning" <ya...@intel.com>.
Thanks for Kai and Emmanuel's good discussion.
>>... keep it simple and use List<String> getIdentiies() at the moment.
This sounds good to me to proceed. Thanks!
Yaning
-----Original Message-----
From: Zheng, Kai
Sent: Monday, July 06, 2015 2:05 PM
To: Xu, Yaning
Subject: FW: How to implement getIdentities(int start, int limit) more effectively
-----Original Message-----
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Monday, July 06, 2015 1:43 PM
To: kerby@directory.apache.org
Subject: RE: How to implement getIdentities(int start, int limit) more effectively
>>... keep it simple and use List<String> getIdentiies() at the moment.
Sounds good to me, thanks!!
>> Cursor already implements Iterable, so if you drink the cool aid fully, you'll get the equivalent to what you propose.
Well, I guess I need some time to absorb the extra benefit of cursor approach. I will be back to this topic when I'm ready, to revisit the API for next release.
Regards,
Kai
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Monday, July 06, 2015 1:35 PM
To: kerby@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 06/07/15 07:06, Zheng, Kai a écrit :
> Yaning,
>
> How about going on with this simple API change, as previously proposed by Stefan? I thought it can server well the same purpose for cursor.
>
> List<String> getIdentities(int start, int limit) => Iterator<String>
> getIdentities()
I'll suggest - again - a different approach :
- either you return a cursor
- or you keep the List<String>
Here is why : Cursor already implements Iterable, so if you drink the cool aid fully, you'll get the equivalent to what you propose.
But if you consider it's still too much work, then, please, keep it simple and use List<String> getIdentiies() at the moment.
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
>>... keep it simple and use List<String> getIdentiies() at the moment.
Sounds good to me, thanks!!
>> Cursor already implements Iterable, so if you drink the cool aid fully, you'll get the equivalent to what you propose.
Well, I guess I need some time to absorb the extra benefit of cursor approach. I will be back to this topic when I'm ready, to revisit the API for next release.
Regards,
Kai
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Monday, July 06, 2015 1:35 PM
To: kerby@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 06/07/15 07:06, Zheng, Kai a écrit :
> Yaning,
>
> How about going on with this simple API change, as previously proposed by Stefan? I thought it can server well the same purpose for cursor.
>
> List<String> getIdentities(int start, int limit) => Iterator<String>
> getIdentities()
I'll suggest - again - a different approach :
- either you return a cursor
- or you keep the List<String>
Here is why : Cursor already implements Iterable, so if you drink the cool aid fully, you'll get the equivalent to what you propose.
But if you consider it's still too much work, then, please, keep it simple and use List<String> getIdentiies() at the moment.
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 06/07/15 07:06, Zheng, Kai a écrit :
> Yaning,
>
> How about going on with this simple API change, as previously proposed by Stefan? I thought it can server well the same purpose for cursor.
>
> List<String> getIdentities(int start, int limit)
> =>
> Iterator<String> getIdentities()
I'll suggest - again - a different approach :
- either you return a cursor
- or you keep the List<String>
Here is why : Cursor already implements Iterable, so if you drink the
cool aid fully, you'll get the equivalent to what you propose.
But if you consider it's still too much work, then, please, keep it
simple and use List<String> getIdentiies() at the moment.
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
Yaning,
How about going on with this simple API change, as previously proposed by Stefan? I thought it can server well the same purpose for cursor.
List<String> getIdentities(int start, int limit)
=>
Iterator<String> getIdentities()
Regards,
Kai
-----Original Message-----
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Monday, July 06, 2015 7:27 AM
To: kerby@directory.apache.org
Subject: RE: How to implement getIdentities(int start, int limit) more effectively
>> The client should *always* get the result sorted. That means teh default cursor implem should sort the result.
OK, I see what you meant. One original problem we were hard at is still there, for some backend lacking support of paged retrieving, we still have to get all the results and then perform the desired/spec-ed sorting.
>> No, not at all. The basic operation is just to fetch all the elements, sort them, and wrap a cursor around it.
Hmm, yes it's easy for you, I meant it's not easy for me. We're not implementing a storage like Mavibot, so I'm thinking something different, maybe more lightweight. Maybe Iteration could be more suitable?
Anyway, let we drill down to the real work, look at the existing sample codes, and see how soon we could be out.
>> Ok, at this point, I would say it's your call. I just expressed my opinion, but I don't want to twist your arm, ...
I don't have any call yet, and just want to speed up the first release, as you see, quite some time passed from our initial collaboration discussed plan more than half a year ago... I also see your opinion, thanks!
So to summarize, let's attempt to get the API defined right in the initial release. Cursor or Iterator ...
Regards,
Kai
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Sunday, July 05, 2015 4:23 PM
To: kerby@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 05/07/15 01:47, Zheng, Kai a écrit :
>>> I do think that the cursor implementation should be responsible for the sorting of data, same thing for the paged search (or to implement it if the backend does not).
> Again, the problem is, if no consistent behavior among the backends, how client would benefit? For example, some backend sort the results, but others don't, then how client know that, regarding the results sorted or not?
The client should *always* get the result sorted. That means teh default cursor implem should sort the result.
>
>>> Here is what I suggest : get the cursor used, no matter what ...
> I'm afraid it's not easy to have a good, well discussed cursor or iterator interface design as you might think. Going this way would need much longer time.
No, not at all. The basic operation is just to fetch all the elements, sort them, and wrap a cursor around it. There is plenty of existing samples in ApacheDS you can copy. I evaluate the work to around half a day, one day with tests.
> The most simple ==List<String> getIdentities()== would be good enough for the initial version, since it's only for internal use and we won't be so lucky to hit huge amounts of principals to store in the backend so soon ..., we can release often after the initial one, right?
Right, but don't miss an oportunity to at least define teh API rigth from day one, even if the underlaying implementation sucks.
Ok, at this point, I would say it's your call. I just expressed my opinion, but I don't want to twist your arm, as the getIdentities() method is implemented in many classes. There is not only one way to catch a bird :-)
Thanks Kai !
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
>> The client should *always* get the result sorted. That means teh default cursor implem should sort the result.
OK, I see what you meant. One original problem we were hard at is still there, for some backend lacking support of paged retrieving, we still have to get all the results and then perform the desired/spec-ed sorting.
>> No, not at all. The basic operation is just to fetch all the elements, sort them, and wrap a cursor around it.
Hmm, yes it's easy for you, I meant it's not easy for me. We're not implementing a storage like Mavibot, so I'm thinking something different, maybe more lightweight. Maybe Iteration could be more suitable?
Anyway, let we drill down to the real work, look at the existing sample codes, and see how soon we could be out.
>> Ok, at this point, I would say it's your call. I just expressed my opinion, but I don't want to twist your arm, ...
I don't have any call yet, and just want to speed up the first release, as you see, quite some time passed from our initial collaboration discussed plan more than half a year ago... I also see your opinion, thanks!
So to summarize, let's attempt to get the API defined right in the initial release. Cursor or Iterator ...
Regards,
Kai
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Sunday, July 05, 2015 4:23 PM
To: kerby@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 05/07/15 01:47, Zheng, Kai a écrit :
>>> I do think that the cursor implementation should be responsible for the sorting of data, same thing for the paged search (or to implement it if the backend does not).
> Again, the problem is, if no consistent behavior among the backends, how client would benefit? For example, some backend sort the results, but others don't, then how client know that, regarding the results sorted or not?
The client should *always* get the result sorted. That means teh default cursor implem should sort the result.
>
>>> Here is what I suggest : get the cursor used, no matter what ...
> I'm afraid it's not easy to have a good, well discussed cursor or iterator interface design as you might think. Going this way would need much longer time.
No, not at all. The basic operation is just to fetch all the elements, sort them, and wrap a cursor around it. There is plenty of existing samples in ApacheDS you can copy. I evaluate the work to around half a day, one day with tests.
> The most simple ==List<String> getIdentities()== would be good enough for the initial version, since it's only for internal use and we won't be so lucky to hit huge amounts of principals to store in the backend so soon ..., we can release often after the initial one, right?
Right, but don't miss an oportunity to at least define teh API rigth from day one, even if the underlaying implementation sucks.
Ok, at this point, I would say it's your call. I just expressed my opinion, but I don't want to twist your arm, as the getIdentities() method is implemented in many classes. There is not only one way to catch a bird :-)
Thanks Kai !
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 05/07/15 01:47, Zheng, Kai a écrit :
>>> I do think that the cursor implementation should be responsible for the sorting of data, same thing for the paged search (or to implement it if the backend does not).
> Again, the problem is, if no consistent behavior among the backends, how client would benefit? For example, some backend sort the results, but others don't, then how client know that, regarding the results sorted or not?
The client should *always* get the result sorted. That means teh default
cursor implem should sort the result.
>
>>> Here is what I suggest : get the cursor used, no matter what ...
> I'm afraid it's not easy to have a good, well discussed cursor or iterator interface design as you might think. Going this way would need much longer time.
No, not at all. The basic operation is just to fetch all the elements,
sort them, and wrap a cursor around it. There is plenty of existing
samples in ApacheDS you can copy. I evaluate the work to around half a
day, one day with tests.
> The most simple ==List<String> getIdentities()== would be good enough for the initial version, since it's only for internal use and we won't be so lucky to hit huge amounts of principals to store in the backend so soon ..., we can release often after the initial one, right?
Right, but don't miss an oportunity to at least define teh API rigth
from day one, even if the underlaying implementation sucks.
Ok, at this point, I would say it's your call. I just expressed my
opinion, but I don't want to twist your arm, as the getIdentities()
method is implemented in many classes. There is not only one way to
catch a bird :-)
Thanks Kai !
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
>> I do think that the cursor implementation should be responsible for the sorting of data, same thing for the paged search (or to implement it if the backend does not).
Again, the problem is, if no consistent behavior among the backends, how client would benefit? For example, some backend sort the results, but others don't, then how client know that, regarding the results sorted or not?
>> Here is what I suggest : get the cursor used, no matter what ...
I'm afraid it's not easy to have a good, well discussed cursor or iterator interface design as you might think. Going this way would need much longer time. The most simple ==List<String> getIdentities()== would be good enough for the initial version, since it's only for internal use and we won't be so lucky to hit huge amounts of principals to store in the backend so soon ..., we can release often after the initial one, right?
Regards,
Kai
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Sunday, July 05, 2015 1:43 AM
To: kerby@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 04/07/15 15:27, Zheng, Kai a écrit :
> Thanks Emmanuel for the further discussion.
>
>>> My percezption is that the getIdentities() method should be made specific, per backend. If a backend supports paged search and sort, fine, iyt can use it. If it's not, then it should use a default implementation. In any cases, the user should not be aware of what's going on behind the curtain.
> Sounds good. One thing is we need a mechanism to let the client be able to aware what's the capabilities a backend is of, paged search, sort and etc. If the client isn't able to aware, how it make use of it.
I don't think it's a good idea. That would make it more complex for the clinet that it needs to be. I do think that the cursor implementation should be responsible for the sorting of data, same thing for the paged search (or to implement it if the backend does not).
>
>>> The best way to acheive this is to have the getIdentities() return a Cursor, which can be handled by any client.
> As I said before, the cursor or iterator (proposed by Stefan) looks good to have. My only question is, could we delay this to next release? You may be noted that it's not so easy for us in short time, we have quite a few backends, and quite a few client calls to use the related API. We should be able to get the most required things done and have the first release sooner and better. Please note backend APIs are internal ones, no worrying for admin and end users if we have.
Ok, I see.
Here is what I suggest : get the cursor used, no matter what, but don't implement specific cursors for backend that support paged search or sorting : instead define a DefaultCursor that do a full search, order the result, and return it. Every backend implementaion will use this DefaultCursor from now on, and after the release, we can improve the code by implementing smarter cursors.
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 04/07/15 15:27, Zheng, Kai a écrit :
> Thanks Emmanuel for the further discussion.
>
>>> My percezption is that the getIdentities() method should be made specific, per backend. If a backend supports paged search and sort, fine, iyt can use it. If it's not, then it should use a default implementation. In any cases, the user should not be aware of what's going on behind the curtain.
> Sounds good. One thing is we need a mechanism to let the client be able to aware what's the capabilities a backend is of, paged search, sort and etc. If the client isn't able to aware, how it make use of it.
I don't think it's a good idea. That would make it more complex for the
clinet that it needs to be. I do think that the cursor implementation
should be responsible for the sorting of data, same thing for the paged
search (or to implement it if the backend does not).
>
>>> The best way to acheive this is to have the getIdentities() return a Cursor, which can be handled by any client.
> As I said before, the cursor or iterator (proposed by Stefan) looks good to have. My only question is, could we delay this to next release? You may be noted that it's not so easy for us in short time, we have quite a few backends, and quite a few client calls to use the related API. We should be able to get the most required things done and have the first release sooner and better. Please note backend APIs are internal ones, no worrying for admin and end users if we have.
Ok, I see.
Here is what I suggest : get the cursor used, no matter what, but don't
implement specific cursors for backend that support paged search or
sorting : instead define a DefaultCursor that do a full search, order
the result, and return it. Every backend implementaion will use this
DefaultCursor from now on, and after the release, we can improve the
code by implementing smarter cursors.
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Emmanuel for the further discussion.
>> My percezption is that the getIdentities() method should be made specific, per backend. If a backend supports paged search and sort, fine, iyt can use it. If it's not, then it should use a default implementation. In any cases, the user should not be aware of what's going on behind the curtain.
Sounds good. One thing is we need a mechanism to let the client be able to aware what's the capabilities a backend is of, paged search, sort and etc. If the client isn't able to aware, how it make use of it.
>> The best way to acheive this is to have the getIdentities() return a Cursor, which can be handled by any client.
As I said before, the cursor or iterator (proposed by Stefan) looks good to have. My only question is, could we delay this to next release? You may be noted that it's not so easy for us in short time, we have quite a few backends, and quite a few client calls to use the related API. We should be able to get the most required things done and have the first release sooner and better. Please note backend APIs are internal ones, no worrying for admin and end users if we have.
>> All in all, define a getIdentities() interface, with an abstract implementatiuon that fetches everything, and specific implementations that bypass this abstract implementation.
Yeah, agree. Again looks good to me.
Would you agree with the plan? Thanks.
Regards,
Kai
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Friday, July 03, 2015 3:53 PM
To: dev@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 03/07/15 08:49, Xu, Yaning a écrit :
> Kai,
>
> Thanks for your suggestions. It's good to me.
My percezption is that the getIdentities() method should be made specific, per backend. If a backend supports paged search and sort, fine, iyt can use it. If it's not, then it should use a default implementation. In any cases, the user should not be aware of what's going on behind the curtain.
The best way to acheive this is to have the getIdentities() return a Cursor, which can be handled by any client. This cursor will allow the user to move forward or backward, regardless of the number of elements in the cursor. You can have a look at the Cursor interface we are using in ApacheDS :
http://svn.apache.org/viewvc/directory/shared/trunk/ldap/model/src/main/java/org/apache/directory/api/ldap/model/cursor/
Side note : in the SQL world, this is the same approach. Once common mistake though in this world, when design (G)UI which expose pages of elements taken froma cursor is to proceed in two steps :
- first do a SELECT COUNT(*) WERE blah
- second, depending on the number of elements, proceed with a single page, or manage pagination locally, with a second request to the server, fetching the real entry.
This is *idiotic* ! Doing that, you make the server do the same work twice : the SELECT COUNT(*) is as costly as the second request, and it's totally useless, unless you really want to expose the number of elements on the screen, which is rare. The rational is that when you do a normal request, the Cursor API allows you to define the number of elements you'll get back on the client side (default to 10, most of the time).
That means the cursor implementation on the client side will cache N elements, no matter what, because the server will send N elements anyway.
All in all, define a getIdentities() interface, with an abstract implementatiuon that fetches everything, and specific implementations that bypass this abstract implementation.
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 03/07/15 08:49, Xu, Yaning a écrit :
> Kai,
>
> Thanks for your suggestions. It's good to me.
My percezption is that the getIdentities() method should be made
specific, per backend. If a backend supports paged search and sort,
fine, iyt can use it. If it's not, then it should use a default
implementation. In any cases, the user should not be aware of what's
going on behind the curtain.
The best way to acheive this is to have the getIdentities() return a
Cursor, which can be handled by any client. This cursor will allow the
user to move forward or backward, regardless of the number of elements
in the cursor. You can have a look at the Cursor interface we are using
in ApacheDS :
http://svn.apache.org/viewvc/directory/shared/trunk/ldap/model/src/main/java/org/apache/directory/api/ldap/model/cursor/
Side note : in the SQL world, this is the same approach. Once common
mistake though in this world, when design (G)UI which expose pages of
elements taken froma cursor is to proceed in two steps :
- first do a SELECT COUNT(*) WERE blah
- second, depending on the number of elements, proceed with a single
page, or manage pagination locally, with a second request to the server,
fetching the real entry.
This is *idiotic* ! Doing that, you make the server do the same work
twice : the SELECT COUNT(*) is as costly as the second request, and it's
totally useless, unless you really want to expose the number of elements
on the screen, which is rare. The rational is that when you do a normal
request, the Cursor API allows you to define the number of elements
you'll get back on the client side (default to 10, most of the time).
That means the cursor implementation on the client side will cache N
elements, no matter what, because the server will send N elements anyway.
All in all, define a getIdentities() interface, with an abstract
implementatiuon that fetches everything, and specific implementations
that bypass this abstract implementation.
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Xu, Yaning" <ya...@intel.com>.
Kai,
Thanks for your suggestions. It's good to me.
Yaning
-----Original Message-----
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Friday, July 03, 2015 2:37 PM
To: Apache Directory Developers List
Subject: RE: How to implement getIdentities(int start, int limit) more effectively
Thanks all for the thoughts and suggestions.
Yaning, your plan looks good to me, easy to go for the initial version. If nobody objects, would you mind proceeding to:
1. Fire an issue to remove the paged getIdentities(offset, limit) and have the non-paged one getIdentities() just to return all entries? I thought it's good to have for our initial release, we would evolve quickly to have new version to replace it.
2. Fire an issue to support cursor in the backend API, targeting future version.
3. Fire another issue to optionally support sort and paged search, targeting future version.
Another idea is, how MIT Kerberos handling this? I know it supports many backends, like SQL and OpenLDAP. Would be good to take a look at it. Thanks.
Regards,
Kai
-----Original Message-----
From: Xu, Yaning [mailto:yaning.xu@intel.com]
Sent: Friday, July 03, 2015 1:43 PM
To: Apache Directory Developers List
Subject: RE: How to implement getIdentities(int start, int limit) more effectively
Thanks Emmanuel and Kiran for the discussion! As you suggested, it would be good to support cursor, with sort and paged search favored. It looks great to me to have. I'm wondering if it's easy to implement the great features for all the existing backends in the first version. I agree the current paged getIdentities API is problematic, as you discussed, because it may be not doable for some backends. On the other hand, there may not be so many principals for us to worry about considering the initial version. Please note even we allow to get all the identities once a time, only principal names are required to return, so memory cost may not be too much. So is it okay that we remove sort and paged getting support from the APIs and just use getIdentities() to get all principal names in the first version, and leave the great features like sort, paged and cursor support for the following version? Hope this plan sounds good to all of us and easy to go. Thanks!
Regards,
Yaning
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Thursday, July 02, 2015 7:56 PM
To: dev@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 02/07/15 13:25, Kiran Ayyagari a écrit :
> On Thu, Jul 2, 2015 at 6:05 PM, Emmanuel Lécharny
> <el...@gmail.com>
> wrote:
>
>> Le 02/07/15 11:14, Kiran Ayyagari a écrit :
>>> Hi Yaning,
>>>
>>> On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
>>>
>>>> Hi Kiran,
>>>>
>>>>
>>>>
>>>> 1. The administrator may need to list all principal names, and
>> then
>>>> use the principal name he is interested in to get the Identity;
>>>>
>>>> 2. There may be too many principals to scan at once, so the system
>>>> may need to get only a part of them at once, as Stefan pointed out;
>>>>
>>>> 3. We need to make Kerby compatible to Krb5, and it has
>> implemented
>>>> this interface as list_principals
>>>>
>> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin
>> _local.html
>>>> ;
>>>>
>>> ic, so here is the case, it is mainly for the kadmin client and I
>>> would prefer if kadmin takes care of sorting and paginating after
>>> retrieving the principals rather than offloading this to backends.
>>>
>>> The primary reasons is not all stores that are used by
>>> IdentityBackends
>> may
>>> support sorting or
>>> pagination, for example not all LDAP servers support sorting and
>>> many database libraries do not support pagination(this is a client
>>> thing).
>>>
>>> I would like to propose an alternative design:
>>>
>>> Add search functionality to the IdentityBackend, i.e it accepts a
>>> search pattern and return a Cursor then kadmin will browse this
>>> cursor and prepares a suitable view of principals.
>>> We can have a search limit in this interface but we don't need a
>>> start position cause the client can navigate using Cursor (when the
>>> cursor implementation supports).
>> I tend to disagree with Kiran, here.
>>
>> If a server does not support 'sort' and 'paged search', then there is
>> no mean to use it as a backend. AFAICT, most of the existing LDAP
>> server on the market support both controls.
>>
> this is not just for LDAP servers, but any of the 'store's that are
> used by Kerby say, there is one using Mavibot, we need to fetch *all*
> and perform a search cause Mavibot doesn't natively offer a
> substring/wildcard search.
True. I was specifcially speaking about LDAP backend, I should have be explicit.
>
>> In any case, don't implement it on the client side : with millions
>> and candidate, it will not only be slow, but it will also eat a hell
>> lot of memory.
>>
> here the client is not just authenticating but is is an admin UI for
> Kerby's database, this client can sort, search and paginate perhaps
> with a wrapper backend that does all the functionality.
>
> My idea is not to enforce this requirement on each backend implementation.
I'd rather suggest that the API expose a Cursor. The backends that support Sort and Paged Search would be favored, and the others, well, they will have to implement the Cursor interface.
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks all for the thoughts and suggestions.
Yaning, your plan looks good to me, easy to go for the initial version. If nobody objects, would you mind proceeding to:
1. Fire an issue to remove the paged getIdentities(offset, limit) and have the non-paged one getIdentities() just to return all entries? I thought it's good to have for our initial release, we would evolve quickly to have new version to replace it.
2. Fire an issue to support cursor in the backend API, targeting future version.
3. Fire another issue to optionally support sort and paged search, targeting future version.
Another idea is, how MIT Kerberos handling this? I know it supports many backends, like SQL and OpenLDAP. Would be good to take a look at it. Thanks.
Regards,
Kai
-----Original Message-----
From: Xu, Yaning [mailto:yaning.xu@intel.com]
Sent: Friday, July 03, 2015 1:43 PM
To: Apache Directory Developers List
Subject: RE: How to implement getIdentities(int start, int limit) more effectively
Thanks Emmanuel and Kiran for the discussion! As you suggested, it would be good to support cursor, with sort and paged search favored. It looks great to me to have. I'm wondering if it's easy to implement the great features for all the existing backends in the first version. I agree the current paged getIdentities API is problematic, as you discussed, because it may be not doable for some backends. On the other hand, there may not be so many principals for us to worry about considering the initial version. Please note even we allow to get all the identities once a time, only principal names are required to return, so memory cost may not be too much. So is it okay that we remove sort and paged getting support from the APIs and just use getIdentities() to get all principal names in the first version, and leave the great features like sort, paged and cursor support for the following version? Hope this plan sounds good to all of us and easy to go. Thanks!
Regards,
Yaning
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Thursday, July 02, 2015 7:56 PM
To: dev@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 02/07/15 13:25, Kiran Ayyagari a écrit :
> On Thu, Jul 2, 2015 at 6:05 PM, Emmanuel Lécharny
> <el...@gmail.com>
> wrote:
>
>> Le 02/07/15 11:14, Kiran Ayyagari a écrit :
>>> Hi Yaning,
>>>
>>> On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
>>>
>>>> Hi Kiran,
>>>>
>>>>
>>>>
>>>> 1. The administrator may need to list all principal names, and
>> then
>>>> use the principal name he is interested in to get the Identity;
>>>>
>>>> 2. There may be too many principals to scan at once, so the system
>>>> may need to get only a part of them at once, as Stefan pointed out;
>>>>
>>>> 3. We need to make Kerby compatible to Krb5, and it has
>> implemented
>>>> this interface as list_principals
>>>>
>> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin
>> _local.html
>>>> ;
>>>>
>>> ic, so here is the case, it is mainly for the kadmin client and I
>>> would prefer if kadmin takes care of sorting and paginating after
>>> retrieving the principals rather than offloading this to backends.
>>>
>>> The primary reasons is not all stores that are used by
>>> IdentityBackends
>> may
>>> support sorting or
>>> pagination, for example not all LDAP servers support sorting and
>>> many database libraries do not support pagination(this is a client
>>> thing).
>>>
>>> I would like to propose an alternative design:
>>>
>>> Add search functionality to the IdentityBackend, i.e it accepts a
>>> search pattern and return a Cursor then kadmin will browse this
>>> cursor and prepares a suitable view of principals.
>>> We can have a search limit in this interface but we don't need a
>>> start position cause the client can navigate using Cursor (when the
>>> cursor implementation supports).
>> I tend to disagree with Kiran, here.
>>
>> If a server does not support 'sort' and 'paged search', then there is
>> no mean to use it as a backend. AFAICT, most of the existing LDAP
>> server on the market support both controls.
>>
> this is not just for LDAP servers, but any of the 'store's that are
> used by Kerby say, there is one using Mavibot, we need to fetch *all*
> and perform a search cause Mavibot doesn't natively offer a
> substring/wildcard search.
True. I was specifcially speaking about LDAP backend, I should have be explicit.
>
>> In any case, don't implement it on the client side : with millions
>> and candidate, it will not only be slow, but it will also eat a hell
>> lot of memory.
>>
> here the client is not just authenticating but is is an admin UI for
> Kerby's database, this client can sort, search and paginate perhaps
> with a wrapper backend that does all the functionality.
>
> My idea is not to enforce this requirement on each backend implementation.
I'd rather suggest that the API expose a Cursor. The backends that support Sort and Paged Search would be favored, and the others, well, they will have to implement the Cursor interface.
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Xu, Yaning" <ya...@intel.com>.
Thanks Emmanuel and Kiran for the discussion! As you suggested, it would be good to support cursor, with sort and paged search favored. It looks great to me to have. I'm wondering if it's easy to implement the great features for all the existing backends in the first version. I agree the current paged getIdentities API is problematic, as you discussed, because it may be not doable for some backends. On the other hand, there may not be so many principals for us to worry about considering the initial version. Please note even we allow to get all the identities once a time, only principal names are required to return, so memory cost may not be too much. So is it okay that we remove sort and paged getting support from the APIs and just use getIdentities() to get all principal names in the first version, and leave the great features like sort, paged and cursor support for the following version? Hope this plan sounds good to all of us and easy to go. Thanks!
Regards,
Yaning
-----Original Message-----
From: Emmanuel Lécharny [mailto:elecharny@gmail.com]
Sent: Thursday, July 02, 2015 7:56 PM
To: dev@directory.apache.org
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Le 02/07/15 13:25, Kiran Ayyagari a écrit :
> On Thu, Jul 2, 2015 at 6:05 PM, Emmanuel Lécharny
> <el...@gmail.com>
> wrote:
>
>> Le 02/07/15 11:14, Kiran Ayyagari a écrit :
>>> Hi Yaning,
>>>
>>> On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
>>>
>>>> Hi Kiran,
>>>>
>>>>
>>>>
>>>> 1. The administrator may need to list all principal names, and
>> then
>>>> use the principal name he is interested in to get the Identity;
>>>>
>>>> 2. There may be too many principals to scan at once, so the system
>>>> may need to get only a part of them at once, as Stefan pointed out;
>>>>
>>>> 3. We need to make Kerby compatible to Krb5, and it has
>> implemented
>>>> this interface as list_principals
>>>>
>> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin
>> _local.html
>>>> ;
>>>>
>>> ic, so here is the case, it is mainly for the kadmin client and I
>>> would prefer if kadmin takes care of sorting and paginating after
>>> retrieving the principals rather than offloading this to backends.
>>>
>>> The primary reasons is not all stores that are used by
>>> IdentityBackends
>> may
>>> support sorting or
>>> pagination, for example not all LDAP servers support sorting and
>>> many database libraries do not support pagination(this is a client
>>> thing).
>>>
>>> I would like to propose an alternative design:
>>>
>>> Add search functionality to the IdentityBackend, i.e it accepts a
>>> search pattern and return a Cursor then kadmin will browse this
>>> cursor and prepares a suitable view of principals.
>>> We can have a search limit in this interface but we don't need a
>>> start position cause the client can navigate using Cursor (when the
>>> cursor implementation supports).
>> I tend to disagree with Kiran, here.
>>
>> If a server does not support 'sort' and 'paged search', then there is
>> no mean to use it as a backend. AFAICT, most of the existing LDAP
>> server on the market support both controls.
>>
> this is not just for LDAP servers, but any of the 'store's that are
> used by Kerby say, there is one using Mavibot, we need to fetch *all*
> and perform a search cause Mavibot doesn't natively offer a
> substring/wildcard search.
True. I was specifcially speaking about LDAP backend, I should have be explicit.
>
>> In any case, don't implement it on the client side : with millions
>> and candidate, it will not only be slow, but it will also eat a hell
>> lot of memory.
>>
> here the client is not just authenticating but is is an admin UI for
> Kerby's database, this client can sort, search and paginate perhaps
> with a wrapper backend that does all the functionality.
>
> My idea is not to enforce this requirement on each backend implementation.
I'd rather suggest that the API expose a Cursor. The backends that support Sort and Paged Search would be favored, and the others, well, they will have to implement the Cursor interface.
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 02/07/15 13:25, Kiran Ayyagari a écrit :
> On Thu, Jul 2, 2015 at 6:05 PM, Emmanuel Lécharny <el...@gmail.com>
> wrote:
>
>> Le 02/07/15 11:14, Kiran Ayyagari a écrit :
>>> Hi Yaning,
>>>
>>> On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
>>>
>>>> Hi Kiran,
>>>>
>>>>
>>>>
>>>> 1. The administrator may need to list all principal names, and
>> then
>>>> use the principal name he is interested in to get the Identity;
>>>>
>>>> 2. There may be too many principals to scan at once, so the system
>>>> may need to get only a part of them at once, as Stefan pointed out;
>>>>
>>>> 3. We need to make Kerby compatible to Krb5, and it has
>> implemented
>>>> this interface as list_principals
>>>>
>> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin_local.html
>>>> ;
>>>>
>>> ic, so here is the case, it is mainly for the kadmin client and I would
>>> prefer if kadmin takes care of
>>> sorting and paginating after retrieving the principals rather than
>>> offloading this to backends.
>>>
>>> The primary reasons is not all stores that are used by IdentityBackends
>> may
>>> support sorting or
>>> pagination, for example not all LDAP servers support sorting and many
>>> database libraries do not
>>> support pagination(this is a client thing).
>>>
>>> I would like to propose an alternative design:
>>>
>>> Add search functionality to the IdentityBackend, i.e it accepts a search
>>> pattern and return a Cursor
>>> then kadmin will browse this cursor and prepares a suitable view of
>>> principals.
>>> We can have a search limit in this interface but we don't need a start
>>> position cause the client can
>>> navigate using Cursor (when the cursor implementation supports).
>> I tend to disagree with Kiran, here.
>>
>> If a server does not support 'sort' and 'paged search', then there is no
>> mean to use it as a backend. AFAICT, most of the existing LDAP server on
>> the market support both controls.
>>
> this is not just for LDAP servers, but any of the 'store's that are used by
> Kerby
> say, there is one using Mavibot, we need to fetch *all* and perform a search
> cause Mavibot doesn't natively offer a substring/wildcard search.
True. I was specifcially speaking about LDAP backend, I should have be
explicit.
>
>> In any case, don't implement it on the client side : with millions and
>> candidate, it will not only be slow, but it will also eat a hell lot of
>> memory.
>>
> here the client is not just authenticating but is is an admin UI for
> Kerby's database,
> this client can sort, search and paginate perhaps with a wrapper backend
> that
> does all the functionality.
>
> My idea is not to enforce this requirement on each backend implementation.
I'd rather suggest that the API expose a Cursor. The backends that
support Sort and Paged Search would be favored, and the others, well,
they will have to implement the Cursor interface.
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Kiran Ayyagari <ka...@apache.org>.
On Thu, Jul 2, 2015 at 6:05 PM, Emmanuel Lécharny <el...@gmail.com>
wrote:
> Le 02/07/15 11:14, Kiran Ayyagari a écrit :
> > Hi Yaning,
> >
> > On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
> >
> >> Hi Kiran,
> >>
> >>
> >>
> >> 1. The administrator may need to list all principal names, and
> then
> >> use the principal name he is interested in to get the Identity;
> >>
> >> 2. There may be too many principals to scan at once, so the system
> >> may need to get only a part of them at once, as Stefan pointed out;
> >>
> >> 3. We need to make Kerby compatible to Krb5, and it has
> implemented
> >> this interface as list_principals
> >>
> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin_local.html
> >> ;
> >>
> > ic, so here is the case, it is mainly for the kadmin client and I would
> > prefer if kadmin takes care of
> > sorting and paginating after retrieving the principals rather than
> > offloading this to backends.
> >
> > The primary reasons is not all stores that are used by IdentityBackends
> may
> > support sorting or
> > pagination, for example not all LDAP servers support sorting and many
> > database libraries do not
> > support pagination(this is a client thing).
> >
> > I would like to propose an alternative design:
> >
> > Add search functionality to the IdentityBackend, i.e it accepts a search
> > pattern and return a Cursor
> > then kadmin will browse this cursor and prepares a suitable view of
> > principals.
> > We can have a search limit in this interface but we don't need a start
> > position cause the client can
> > navigate using Cursor (when the cursor implementation supports).
>
> I tend to disagree with Kiran, here.
>
> If a server does not support 'sort' and 'paged search', then there is no
> mean to use it as a backend. AFAICT, most of the existing LDAP server on
> the market support both controls.
>
this is not just for LDAP servers, but any of the 'store's that are used by
Kerby
say, there is one using Mavibot, we need to fetch *all* and perform a search
cause Mavibot doesn't natively offer a substring/wildcard search.
>
> In any case, don't implement it on the client side : with millions and
> candidate, it will not only be slow, but it will also eat a hell lot of
> memory.
>
here the client is not just authenticating but is is an admin UI for
Kerby's database,
this client can sort, search and paginate perhaps with a wrapper backend
that
does all the functionality.
My idea is not to enforce this requirement on each backend implementation.
--
Kiran Ayyagari
http://keydap.com
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 02/07/15 11:14, Kiran Ayyagari a écrit :
> Hi Yaning,
>
> On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
>
>> Hi Kiran,
>>
>>
>>
>> 1. The administrator may need to list all principal names, and then
>> use the principal name he is interested in to get the Identity;
>>
>> 2. There may be too many principals to scan at once, so the system
>> may need to get only a part of them at once, as Stefan pointed out;
>>
>> 3. We need to make Kerby compatible to Krb5, and it has implemented
>> this interface as list_principals
>> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin_local.html
>> ;
>>
> ic, so here is the case, it is mainly for the kadmin client and I would
> prefer if kadmin takes care of
> sorting and paginating after retrieving the principals rather than
> offloading this to backends.
>
> The primary reasons is not all stores that are used by IdentityBackends may
> support sorting or
> pagination, for example not all LDAP servers support sorting and many
> database libraries do not
> support pagination(this is a client thing).
>
> I would like to propose an alternative design:
>
> Add search functionality to the IdentityBackend, i.e it accepts a search
> pattern and return a Cursor
> then kadmin will browse this cursor and prepares a suitable view of
> principals.
> We can have a search limit in this interface but we don't need a start
> position cause the client can
> navigate using Cursor (when the cursor implementation supports).
I tend to disagree with Kiran, here.
If a server does not support 'sort' and 'paged search', then there is no
mean to use it as a backend. AFAICT, most of the existing LDAP server on
the market support both controls.
In any case, don't implement it on the client side : with millions and
candidate, it will not only be slow, but it will also eat a hell lot of
memory.
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Kiran Ayyagari <ka...@apache.org>.
Hi Yaning,
On Thu, Jul 2, 2015 at 4:28 PM, Xu, Yaning <ya...@intel.com> wrote:
> Hi Kiran,
>
>
>
> 1. The administrator may need to list all principal names, and then
> use the principal name he is interested in to get the Identity;
>
> 2. There may be too many principals to scan at once, so the system
> may need to get only a part of them at once, as Stefan pointed out;
>
> 3. We need to make Kerby compatible to Krb5, and it has implemented
> this interface as list_principals
> http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin_local.html
> ;
>
ic, so here is the case, it is mainly for the kadmin client and I would
prefer if kadmin takes care of
sorting and paginating after retrieving the principals rather than
offloading this to backends.
The primary reasons is not all stores that are used by IdentityBackends may
support sorting or
pagination, for example not all LDAP servers support sorting and many
database libraries do not
support pagination(this is a client thing).
I would like to propose an alternative design:
Add search functionality to the IdentityBackend, i.e it accepts a search
pattern and return a Cursor
then kadmin will browse this cursor and prepares a suitable view of
principals.
We can have a search limit in this interface but we don't need a start
position cause the client can
navigate using Cursor (when the cursor implementation supports).
thanks for the link
>
> Regards,
>
>
>
> Yaning
>
>
>
>
>
> *From:* Kiran Ayyagari [mailto:kayyagari@apache.org]
> *Sent:* Thursday, July 02, 2015 3:59 PM
> *To:* Apache Directory Developers List
> *Subject:* Re: How to implement getIdentities(int start, int limit) more
> effectively
>
>
>
> Hi Yaning,
>
> where in the server or client are we going to need this interface, iow,
> can you describe the usecase
>
> that requires this getIdentities(s,l) method.
>
> (I just want to understand the need for this method before we dive into
> implementing it)
>
>
>
> On Thu, Jul 2, 2015 at 3:39 PM, Xu, Yaning <ya...@intel.com> wrote:
>
> Hi all,
>
>
>
> As we have discussed https://issues.apache.org/jira/browse/DIRKRB-295 , IdentityService.getIdentities() should be removed. However, to implement getIdentities(int start, int limit), which returns a sorted result, I have to get all identities from the ldap server and then sort them. Is it possible for the Ldap server to return a sorted result for the client connection?
>
>
>
> In [1], I see a test about Page Search on client side testPagedSearchWrongCookie(), and it uses pagedSearchControl.setSize(3) to set the page size, the problem are:
>
>
>
> *how can I get a search result based on the start index and pageSize, like I may use getIdentities(int start, int limit) to return a result;
>
> *how can I get a sorted result from the server, so that I don’t have to get all the entries and then sort them to get the result;
>
>
>
> [1] https://svn.apache.org/repos/asf/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PagedSearchIT.java
>
>
>
>
>
> public List<String> getIdentities(int start, int limit) {
> List<String> identities = getIdentities();
>
> if (limit == -1) {
> return identities;
> }
>
> return getIdentities().subList(start, start + limit);
> }
>
>
> private List<String> getIdentities() {
> List<String> identityNames = new ArrayList<>();
> EntryCursor cursor;
> Entry entry;
> try {
> cursor = connection.search( getConfig().getString("base_dn"),
> "(objectclass=*)", SearchScope.ONELEVEL, KerberosAttribute.KRB5_PRINCIPAL_NAME_AT);
> if (cursor == null) {
> return null;
> }
> while (cursor.next()) {
> entry = cursor.get();
> identityNames.add(entry.get(KerberosAttribute.KRB5_PRINCIPAL_NAME_AT).getString());
> }
> cursor.close();
> Collections.sort(identityNames);
> } catch (LdapException e) {
> e.printStackTrace();
> } catch (CursorException e) {
> e.printStackTrace();
> }
> return identityNames;
> }
>
>
>
> Regards,
>
>
>
> Yaning
>
>
>
>
>
>
> --
>
> Kiran Ayyagari
> http://keydap.com
>
--
Kiran Ayyagari
http://keydap.com
RE: How to implement getIdentities(int start, int limit) more
effectively
Posted by "Xu, Yaning" <ya...@intel.com>.
Hi Kiran,
1. The administrator may need to list all principal names, and then use the principal name he is interested in to get the Identity;
2. There may be too many principals to scan at once, so the system may need to get only a part of them at once, as Stefan pointed out;
3. We need to make Kerby compatible to Krb5, and it has implemented this interface as list_principals http://web.mit.edu/KERBEROS/krb5-1.12/doc/admin/admin_commands/kadmin_local.html ;
Regards,
Yaning
From: Kiran Ayyagari [mailto:kayyagari@apache.org]
Sent: Thursday, July 02, 2015 3:59 PM
To: Apache Directory Developers List
Subject: Re: How to implement getIdentities(int start, int limit) more effectively
Hi Yaning,
where in the server or client are we going to need this interface, iow, can you describe the usecase
that requires this getIdentities(s,l) method.
(I just want to understand the need for this method before we dive into implementing it)
On Thu, Jul 2, 2015 at 3:39 PM, Xu, Yaning <ya...@intel.com>> wrote:
Hi all,
As we have discussed https://issues.apache.org/jira/browse/DIRKRB-295 , IdentityService.getIdentities() should be removed. However, to implement getIdentities(int start, int limit), which returns a sorted result, I have to get all identities from the ldap server and then sort them. Is it possible for the Ldap server to return a sorted result for the client connection?
In [1], I see a test about Page Search on client side testPagedSearchWrongCookie(), and it uses pagedSearchControl.setSize(3) to set the page size, the problem are:
*how can I get a search result based on the start index and pageSize, like I may use getIdentities(int start, int limit) to return a result;
*how can I get a sorted result from the server, so that I don’t have to get all the entries and then sort them to get the result;
[1] https://svn.apache.org/repos/asf/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PagedSearchIT.java
public List<String> getIdentities(int start, int limit) {
List<String> identities = getIdentities();
if (limit == -1) {
return identities;
}
return getIdentities().subList(start, start + limit);
}
private List<String> getIdentities() {
List<String> identityNames = new ArrayList<>();
EntryCursor cursor;
Entry entry;
try {
cursor = connection.search( getConfig().getString("base_dn"),
"(objectclass=*)", SearchScope.ONELEVEL, KerberosAttribute.KRB5_PRINCIPAL_NAME_AT);
if (cursor == null) {
return null;
}
while (cursor.next()) {
entry = cursor.get();
identityNames.add(entry.get(KerberosAttribute.KRB5_PRINCIPAL_NAME_AT).getString());
}
cursor.close();
Collections.sort(identityNames);
} catch (LdapException e) {
e.printStackTrace();
} catch (CursorException e) {
e.printStackTrace();
}
return identityNames;
}
Regards,
Yaning
--
Kiran Ayyagari
http://keydap.com
Re: How to implement getIdentities(int start, int limit) more effectively
Posted by Kiran Ayyagari <ka...@apache.org>.
Hi Yaning,
where in the server or client are we going to need this interface, iow,
can you describe the usecase
that requires this getIdentities(s,l) method.
(I just want to understand the need for this method before we dive into
implementing it)
On Thu, Jul 2, 2015 at 3:39 PM, Xu, Yaning <ya...@intel.com> wrote:
> Hi all,
>
>
>
> As we have discussed https://issues.apache.org/jira/browse/DIRKRB-295 , IdentityService.getIdentities() should be removed. However, to implement getIdentities(int start, int limit), which returns a sorted result, I have to get all identities from the ldap server and then sort them. Is it possible for the Ldap server to return a sorted result for the client connection?
>
>
>
> In [1], I see a test about Page Search on client side testPagedSearchWrongCookie(), and it uses pagedSearchControl.setSize(3) to set the page size, the problem are:
>
>
>
> *how can I get a search result based on the start index and pageSize, like I may use getIdentities(int start, int limit) to return a result;
>
> *how can I get a sorted result from the server, so that I don’t have to get all the entries and then sort them to get the result;
>
>
>
> [1] https://svn.apache.org/repos/asf/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PagedSearchIT.java
>
>
>
>
>
> public List<String> getIdentities(int start, int limit) {
> List<String> identities = getIdentities();
>
> if (limit == -1) {
> return identities;
> }
>
> return getIdentities().subList(start, start + limit);
> }
>
>
> private List<String> getIdentities() {
> List<String> identityNames = new ArrayList<>();
> EntryCursor cursor;
> Entry entry;
> try {
> cursor = connection.search( getConfig().getString("base_dn"),
> "(objectclass=*)", SearchScope.ONELEVEL, KerberosAttribute.KRB5_PRINCIPAL_NAME_AT);
> if (cursor == null) {
> return null;
> }
> while (cursor.next()) {
> entry = cursor.get();
> identityNames.add(entry.get(KerberosAttribute.KRB5_PRINCIPAL_NAME_AT).getString());
> }
> cursor.close();
> Collections.sort(identityNames);
> } catch (LdapException e) {
> e.printStackTrace();
> } catch (CursorException e) {
> e.printStackTrace();
> }
> return identityNames;
> }
>
>
>
> Regards,
>
>
>
> Yaning
>
>
>
--
Kiran Ayyagari
http://keydap.com