You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@archiva.apache.org by Jason van Zyl <ja...@maven.org> on 2006/09/19 15:13:39 UTC
Separating the Indices
Hi,
I'm currently using the md5 information in the index to lookup
artifacts but the index as a whole is 175mb which is a little
unwieldy. Anyone mind if I take a shot at separating the indices and
coming up with a plan for segmenting them on daily, weekly, monthly
chunks and having lucene aggregate them again? This is somethign that
I would like for the build conversion tool I'm writing and I'll need
it next week when I work with Milos on the embedder. The index file
needed by IDEs for selecting dependencies is fairly small compressed,
trying to pull down a compressed archive of 17mb (bzip2, but growing)
is a bit much.
Jason van Zyl
jason@maven.org
Re: Separating the Indices
Posted by Jason van Zyl <ja...@maven.org>.
On 19 Sep 06, at 3:17 PM 19 Sep 06, Brett Porter wrote:
> IT's not hooked up, but there is a "minimal" index too that should
> be much smaller. It's the original one used by the eclipse plugin.
>
> But no objections from me if you want to work on the other one.
>
Cool, I'll work with Eugene as he had some good ideas about how the
indices could be generated and staged so that they are most useful
for tools.
> - Brett
>
> On 19/09/2006, at 11:13 PM, Jason van Zyl wrote:
>
>> Hi,
>>
>> I'm currently using the md5 information in the index to lookup
>> artifacts but the index as a whole is 175mb which is a little
>> unwieldy. Anyone mind if I take a shot at separating the indices
>> and coming up with a plan for segmenting them on daily, weekly,
>> monthly chunks and having lucene aggregate them again? This is
>> somethign that I would like for the build conversion tool I'm
>> writing and I'll need it next week when I work with Milos on the
>> embedder. The index file needed by IDEs for selecting dependencies
>> is fairly small compressed, trying to pull down a compressed
>> archive of 17mb (bzip2, but growing) is a bit much.
>>
>> Jason van Zyl
>> jason@maven.org
>>
>>
>
Jason van Zyl
jason@maven.org
Re: Separating the Indices
Posted by Brett Porter <br...@apache.org>.
IT's not hooked up, but there is a "minimal" index too that should be
much smaller. It's the original one used by the eclipse plugin.
But no objections from me if you want to work on the other one.
- Brett
On 19/09/2006, at 11:13 PM, Jason van Zyl wrote:
> Hi,
>
> I'm currently using the md5 information in the index to lookup
> artifacts but the index as a whole is 175mb which is a little
> unwieldy. Anyone mind if I take a shot at separating the indices
> and coming up with a plan for segmenting them on daily, weekly,
> monthly chunks and having lucene aggregate them again? This is
> somethign that I would like for the build conversion tool I'm
> writing and I'll need it next week when I work with Milos on the
> embedder. The index file needed by IDEs for selecting dependencies
> is fairly small compressed, trying to pull down a compressed
> archive of 17mb (bzip2, but growing) is a bit much.
>
> Jason van Zyl
> jason@maven.org
>
>