You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by eric tse <hf...@gmail.com> on 2014/01/30 16:13:58 UTC
Ldif sort and Ldif split
Hello,
Do you guys have an apache brand of tools that can
do something like
sort ldif
&
split ldif
?????
Thanks and regards,
Eric
Re: Ldif sort and Ldif split
Posted by Emmanuel Lecharny <el...@apache.org>.
Eric, if you come with some kind of tool that works, we would appreciate a
contribution to the project...
Le 30 janv. 2014 16:39, "eric tse" <hf...@gmail.com> a écrit :
> Thanks a lot :)
>
>
> On Thu, Jan 30, 2014 at 8:33 AM, Kiran Ayyagari <ka...@apache.org>wrote:
>
>>
>>
>>
>> On Thu, Jan 30, 2014 at 8:43 PM, eric tse <hf...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> Do you guys have an apache brand of tools that can
>>>
>>> do something like
>>>
>>> sort ldif
>>> &
>>> split ldif
>>>
>>> ?????
>>>
>>> I have this functionality as part of a bulk load tool that am creating
>> for Mavibot (the new ApacheDS backend)
>> but this code is not yet committed.
>>
>> I suggest you use the DnComparator[1] to sort the entries.
>>
>> The way I would solve this issue is:
>>
>> 1. instantiate LdifReader
>> 2. instantiate a TreeSet<DnComparator> S
>> 3. for each LDIF entry E
>> a. store E into S
>> b. if size of S >= some threshold count N
>> b1. save all the entries in S to a LDIF file
>> b2. clear S
>> 4. merge all the entries present in each sorted LDIF file.
>>
>> It will be much easier if you use a BTree from Mavibot or JDBM this will
>> avoid the need to merge
>> and also no need to use TreeSet.
>>
>> [1]
>> https://svn.apache.org/repos/asf/directory/shared/trunk/ldap/model/src/main/java/org/apache/directory/api/ldap/model/schema/comparators/DnComparator.java
>>
>>> Thanks and regards,
>>> Eric
>>>
>>
>>
>>
>> --
>> Kiran Ayyagari
>> http://keydap.com
>>
>
>
Re: Ldif sort and Ldif split
Posted by eric tse <hf...@gmail.com>.
Thanks a lot :)
On Thu, Jan 30, 2014 at 8:33 AM, Kiran Ayyagari <ka...@apache.org>wrote:
>
>
>
> On Thu, Jan 30, 2014 at 8:43 PM, eric tse <hf...@gmail.com> wrote:
>
>> Hello,
>>
>> Do you guys have an apache brand of tools that can
>>
>> do something like
>>
>> sort ldif
>> &
>> split ldif
>>
>> ?????
>>
>> I have this functionality as part of a bulk load tool that am creating
> for Mavibot (the new ApacheDS backend)
> but this code is not yet committed.
>
> I suggest you use the DnComparator[1] to sort the entries.
>
> The way I would solve this issue is:
>
> 1. instantiate LdifReader
> 2. instantiate a TreeSet<DnComparator> S
> 3. for each LDIF entry E
> a. store E into S
> b. if size of S >= some threshold count N
> b1. save all the entries in S to a LDIF file
> b2. clear S
> 4. merge all the entries present in each sorted LDIF file.
>
> It will be much easier if you use a BTree from Mavibot or JDBM this will
> avoid the need to merge
> and also no need to use TreeSet.
>
> [1]
> https://svn.apache.org/repos/asf/directory/shared/trunk/ldap/model/src/main/java/org/apache/directory/api/ldap/model/schema/comparators/DnComparator.java
>
>> Thanks and regards,
>> Eric
>>
>
>
>
> --
> Kiran Ayyagari
> http://keydap.com
>
Re: Ldif sort and Ldif split
Posted by Kiran Ayyagari <ka...@apache.org>.
On Thu, Jan 30, 2014 at 8:43 PM, eric tse <hf...@gmail.com> wrote:
> Hello,
>
> Do you guys have an apache brand of tools that can
>
> do something like
>
> sort ldif
> &
> split ldif
>
> ?????
>
> I have this functionality as part of a bulk load tool that am creating for
Mavibot (the new ApacheDS backend)
but this code is not yet committed.
I suggest you use the DnComparator[1] to sort the entries.
The way I would solve this issue is:
1. instantiate LdifReader
2. instantiate a TreeSet<DnComparator> S
3. for each LDIF entry E
a. store E into S
b. if size of S >= some threshold count N
b1. save all the entries in S to a LDIF file
b2. clear S
4. merge all the entries present in each sorted LDIF file.
It will be much easier if you use a BTree from Mavibot or JDBM this will
avoid the need to merge
and also no need to use TreeSet.
[1]
https://svn.apache.org/repos/asf/directory/shared/trunk/ldap/model/src/main/java/org/apache/directory/api/ldap/model/schema/comparators/DnComparator.java
> Thanks and regards,
> Eric
>
--
Kiran Ayyagari
http://keydap.com