purpose of repository configuration in the database?

[Brett Porter <br...@apache.org>]

Hi,

Can someone explain the need for having the repository configuration  
stored in the database?

I've observed that:
- it is only saved there from the configuration files
- it is never modified, nor does it store additional information than  
in the configuration
- it is never removed
- it does not appear to be referenced from any other jdo queries  
(except some constraint classes that in turn are not used)
- some of the fields are never referenced (only the id/name/url are)

We do a lot of looking up the list of repositories from the database,  
which isn't as efficient as using the equivalent stuff already in  
memory from the configuration. Is there any reason not to make a few  
substitutions there?

Thanks,
Brett

Re: purpose of repository configuration in the database?

[Brett Porter <br...@apache.org>]

On 10/09/2007, at 1:10 PM, Joakim Erdfelt wrote:

> Wendy Smoak wrote:
>> On 9/9/07, Joakim Erdfelt <jo...@erdfelt.com> wrote:
>>
>>
>>> The database was always viewed as *the* source for all  
>>> information that
>>> is pertinent to Archiva.
>>> The configuration pieces were viewed not as a model or  
>>> datasource, but a
>>> bootstrap to take a configuration from disk, and populate the  
>>> database.
>>>
>>
>> ... viewed that way by whom?  Not me!  I consider archiva.xml *the*
>> source for configuration.
>>
>> I need to be able to pre-configure _everything_ in plain text, so  
>> at a
>> minimum it needs to keep dumping the information to archiva.xml on
>> shutdown and reading it back in on startup.
>>
>>
>
> I think you misunderstand what I said.
>
> Ideally, the information in the DB is what is used at runtime.
> And the information provided by the configuration components is just
> used to bootstrap / startup the app, not as an ongoing source for the
> information while the application is running.

I'm confused, what is the ongoing source of the information (and what  
is sourcing it) while the application is running?

Currently, the configuration components are what is used by  
everything to determine the config, and that comes from in memory.

And I thought you only said the database was for reporting - which is  
not implemented yet - and was the other part of this thread?

Sorry for all the dense questions :)

- Brett

--
Brett Porter - brett@apache.org
Blog: http://www.devzuz.org/blogs/bporter/

Re: purpose of repository configuration in the database?

[Joakim Erdfelt <jo...@erdfelt.com>]

Wendy Smoak wrote:
> On 9/9/07, Joakim Erdfelt <jo...@erdfelt.com> wrote:
>
>   
>> The database was always viewed as *the* source for all information that
>> is pertinent to Archiva.
>> The configuration pieces were viewed not as a model or datasource, but a
>> bootstrap to take a configuration from disk, and populate the database.
>>     
>
> ... viewed that way by whom?  Not me!  I consider archiva.xml *the*
> source for configuration.
>
> I need to be able to pre-configure _everything_ in plain text, so at a
> minimum it needs to keep dumping the information to archiva.xml on
> shutdown and reading it back in on startup.
>
>   

I think you misunderstand what I said.

Ideally, the information in the DB is what is used at runtime.
And the information provided by the configuration components is just
used to bootstrap / startup the app, not as an ongoing source for the
information while the application is running.
The filesystem based configuration is in no danger of going away or
being replaced, it is too important for those kinds of sinful thoughts.

- Joakim

Re: purpose of repository configuration in the database?

[Wendy Smoak <ws...@gmail.com>]

On 9/9/07, Joakim Erdfelt <jo...@erdfelt.com> wrote:

> The database was always viewed as *the* source for all information that
> is pertinent to Archiva.
> The configuration pieces were viewed not as a model or datasource, but a
> bootstrap to take a configuration from disk, and populate the database.

... viewed that way by whom?  Not me!  I consider archiva.xml *the*
source for configuration.

I need to be able to pre-configure _everything_ in plain text, so at a
minimum it needs to keep dumping the information to archiva.xml on
shutdown and reading it back in on startup.

-- 
Wendy

Re: purpose of repository configuration in the database?

[Brett Porter <br...@apache.org>]

On 10/09/2007, at 7:12 AM, Joakim Erdfelt wrote:
> Brett Porter wrote:
>> Joakim - any thoughts on these 2 additional questions? I'd like to
>> finish tidying this section up soon.
>>
>> Cheers,
>> Brett
>
> Not sure what constitutes the 2 additional questions.

They were the ones with the quesiton marks ;p

Here's what I think you're responses are from your mail:

>> I didn't want to have the merge contents from multiple sources in  
>> order to produce a report.
>>
>>
>
> Makes sense, but I can't see where it is actually used?

I think you covered this in your mail - it doesn't.

>> And the structure of the plexus-registry basically forced me into  
>> using the configuration approach over the db approach.
>>
>
> how so?

I still don't really understand what you are getting at here, but we  
have to be able to store configuration outside the database.


> The reporting would use the database.

Given the use cases below, I don't see any reason why the reporting  
couldn't match the configuration model to the fields in the database.  
But let's cross that bridge when we come to it.

I'll double check the use. At first glance it didn't appear to be  
working properly before, so I intended to comment it out until it was  
used.

Thanks,
Brett

--
Brett Porter - brett@apache.org
Blog: http://www.devzuz.org/blogs/bporter/

Re: purpose of repository configuration in the database?

[Joakim Erdfelt <jo...@erdfelt.com>]

Brett Porter wrote:
> Joakim - any thoughts on these 2 additional questions? I'd like to
> finish tidying this section up soon.
>
> Cheers,
> Brett

Not sure what constitutes the 2 additional questions.
But here's the background on the decision.

The database was always viewed as *the* source for all information that
is pertinent to Archiva.
The configuration pieces were viewed not as a model or datasource, but a
bootstrap to take a configuration from disk, and populate the database.

The reporting would use the database.

The repositories in the database are intentionally kept neutral, so that
the future consumer to auto-adjust embedded <repositories> sections
within pom files to the Archiva repository itself.

Yes, currently the database tracks more fields than are used.
But that will change rapidly post Archiva 1.0, when the auto-repo-adjust
consumer kicks in and has to start tracking *all* repo ids to the map-to
id specified in a configuration screen somwhere (that's the primary
reason it was postponed, the need for a configuration screen)

Also, the field for the source of the entry in the database will start
to take on a more important role, as it can identify if it was manual,
from configuration, or from a pom file. (some examples)

I am against removing of the repository information in the database, as
this will needlessly complicate reporting in the future.
I am for removing the need for a configuration model, and would prefer
that the database persistence and the configuration pieces utilize the
same model.

- Joakim

Re: purpose of repository configuration in the database?

[Brett Porter <br...@apache.org>]

Joakim - any thoughts on these 2 additional questions? I'd like to  
finish tidying this section up soon.

Cheers,
Brett

On 21/08/2007, at 1:05 PM, Brett Porter wrote:

>
> On 21/08/2007, at 1:03 PM, Joakim Erdfelt wrote:
>
>> Simple: Reporting.
>>
>> I didn't want to have the merge contents from multiple sources in  
>> order to produce a report.
>>
>
> Makes sense, but I can't see where it is actually used?
>
>> And the structure of the plexus-registry basically forced me into  
>> using the configuration approach over the db approach.
>
> how so?
>
> - Brett
>
>>
>> - Joakim
>>
>> Brett Porter wrote:
>>> Hi,
>>>
>>> Can someone explain the need for having the repository  
>>> configuration stored in the database?
>>>
>>> I've observed that:
>>> - it is only saved there from the configuration files
>>> - it is never modified, nor does it store additional information  
>>> than in the configuration
>>> - it is never removed
>>> - it does not appear to be referenced from any other jdo queries  
>>> (except some constraint classes that in turn are not used)
>>> - some of the fields are never referenced (only the id/name/url are)
>>>
>>> We do a lot of looking up the list of repositories from the  
>>> database, which isn't as efficient as using the equivalent stuff  
>>> already in memory from the configuration. Is there any reason not  
>>> to make a few substitutions there?
>>>
>>> Thanks,
>>> Brett
>>>

--
Brett Porter - brett@apache.org
Blog: http://www.devzuz.org/blogs/bporter/

Re: purpose of repository configuration in the database?

[Brett Porter <br...@apache.org>]

On 21/08/2007, at 1:03 PM, Joakim Erdfelt wrote:

> Simple: Reporting.
>
> I didn't want to have the merge contents from multiple sources in  
> order to produce a report.
>

Makes sense, but I can't see where it is actually used?

> And the structure of the plexus-registry basically forced me into  
> using the configuration approach over the db approach.

how so?

- Brett

>
> - Joakim
>
> Brett Porter wrote:
>> Hi,
>>
>> Can someone explain the need for having the repository  
>> configuration stored in the database?
>>
>> I've observed that:
>> - it is only saved there from the configuration files
>> - it is never modified, nor does it store additional information  
>> than in the configuration
>> - it is never removed
>> - it does not appear to be referenced from any other jdo queries  
>> (except some constraint classes that in turn are not used)
>> - some of the fields are never referenced (only the id/name/url are)
>>
>> We do a lot of looking up the list of repositories from the  
>> database, which isn't as efficient as using the equivalent stuff  
>> already in memory from the configuration. Is there any reason not  
>> to make a few substitutions there?
>>
>> Thanks,
>> Brett
>>

Re: purpose of repository configuration in the database?

[Joakim Erdfelt <jo...@erdfelt.com>]

Simple: Reporting.

I didn't want to have the merge contents from multiple sources in order 
to produce a report.
And the structure of the plexus-registry basically forced me into using 
the configuration approach over the db approach.

- Joakim

Brett Porter wrote:
> Hi,
>
> Can someone explain the need for having the repository configuration 
> stored in the database?
>
> I've observed that:
> - it is only saved there from the configuration files
> - it is never modified, nor does it store additional information than 
> in the configuration
> - it is never removed
> - it does not appear to be referenced from any other jdo queries 
> (except some constraint classes that in turn are not used)
> - some of the fields are never referenced (only the id/name/url are)
>
> We do a lot of looking up the list of repositories from the database, 
> which isn't as efficient as using the equivalent stuff already in 
> memory from the configuration. Is there any reason not to make a few 
> substitutions there?
>
> Thanks,
> Brett
>