You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Henri Yandell <ba...@generationjava.com> on 2004/11/23 16:54:24 UTC
Embedding Eve Was: To release or not to release?
On Mon, 22 Nov 2004, Martin Wegner wrote:
>> We definitely have embeddability in mind. The core stuff will be POJO
>> and we'll provide, or help with, wrappers for well-known containers.
>
> Great. What I hope happens is that once Eve is working and is embeddable
> that the Jarkarta teams will start to think about moving configuration
> information from static XML/property files to Eve. By doing so we would
Hope so, I want to kill simple-jndi, which is what I currently use as a
way to embed a JNDI server for local configuration. Though, a big part is
that it sits on top of static XML/property files.
What kind of files would you make for Eve? My todo list has: attach
commons-configuration to directory-naming, though I've no idea whether
it's workable. Does Eve have this stuff already?
Hen
Naming and Eve provider pros and cons (was (Re: Embedding Eve Was:
To release or not to release?))
Posted by Alex Karasulu <ao...@bellsouth.net>.
Brett Porter wrote:
> Phil Steitz wrote:
>
>> From the various recent posts, it seems we have two basic use cases
>> here and possibly two solutions for one of them -- maybe not a bad
>> thing, but something to think about.
>>
>> 1) Centralized access to configuration information and object
>> references for physically distributed systems
>>
>> 2) Bootstrapping and providing local, in-memory access to
>> configuration / object references based on information in local flat
>> files.
>>
>> In Java, we want to use the JNDI client API in both cases. 1) is a
>> classic application for Directory Services and certainly part of the
>> motivation for Eve. For 2) either embedded Eve or [naming] will
>> work. I am interested in understanding the pros and cons of these two
>> approaches. We may want to merge them in some way (i.e., modify
>> [naming] to just front embedded Eve). Does this make sense?
>
Let's look at the pro's and con's this will be a good exercise:
I think naming and Eve offer two alternatives for different limits
within the problem space. Each definately has some advantage over the
other depending on where we are in the spectrum. An with that said they
compliment each other nicely.
** Secondary verses Primary Storage **
First off Eve is a disk based solution with caching whereas from my
understanding Naming is in memory. Now as memory is getting cheaper and
cheaper (bigger and bigger) this point is becoming moot. However today
Eve's JDBM based database is better for storing large datasets. We
could also build different database for Eve too like ones in memory to
make this point even more moot.
** Search Performance **
There will be a point where Eve's JDBM database will overtake Naming
when it comes to lookup or search performance. This probably will
happen when the data set is very large on the order of 100K to 1M
items. Basically Eve's JDBM database uses B+Trees to index into
serialized objects in a master table. This lookup takes relatively
constant time with a balanced tree. Naming from my understanding would
use in memory structures to find objects like HashMaps. Also I don't
know if Naming supports search to find objects within an entire
subtree. If Naming supported search using in memory BTree's as indices
then there would be no advantage to using Eve when it comes to search
performance.
** Replicated - Distributed - Language Independent LDAP Store **
If you're looking to replicate your data, access and manage it via LDAP
than Eve is for you. Plus this is a language independent mechanism.
Naming would no be language independent as it is tied to JNDI and Java
obviously. Plus the access name syntaxes are totally different. One
uses LDAP relative and distinguished names whereas the namespace of the
other is specific to java:enc which uses a file path like notation for
addressing objects in the namespace. This is usually more intuitive to
those dealing with simple applications where the applications know
exactly where their stuff is in the tree and search is not needed.
** Schema based Constraints **
Naming does not have any schema driven constraints outside of the Java
language. LDAP uses schema much like a relational database schema to
constrain the entities stored within the directory. Although objects
can be serialized into an LDAP directory this is not the primary means
to store objects. Usually u want to parameterize the object into atomic
fields which can be made searchable instead of storing blobs of
serialized objects. This makes for faster searches which are not tied
to Java - other apps can use the same data in languages besides Java.
C O N C L U S I O N S
Really Eve is a heavy weight store when compared with Naming and has
search capabilities using LDAP filters. It is for language independent
environments where the same data may be accessed by non-Java clients
via LDAP. It allows data to be managed and constrained using external
tools. It is more of an enterprise scale solution where as Naming may
not be.
It's namespace syntax is much more complex than Naming which makes the
use of Naming for rapidly building apps much more attractive. If you're
just starting out, and don't know what your object domain is. Or you're
building a new application that is only going to be based on Java.
Further more you don't know what interactions are going to be required
with the environment and other applications the don't bother using Eve
for configuration. Naming is a much better tool for rapidly building
applications without worrying about LDAP and schema and how to represent
your Java objects. Just use Naming to just store your objects for your
configuration and be done with it.
In general I'd recommend using Naming over Eve except when you're
problem space is very well understood and you need to have your
configuration information managed and constrained within an enterprise.
When you jump to this level then Eve is perfect for you. Plus you can
switch between hitting a remote LDAP directory and embedding Eve without
code changes so that's a big win. I think this last recommendation goes
in line with the one I have for those folks who buid database driven
apps: don't worry about a database or a schema just focus on getting the
object model right and use an in memory data store that does not require
thinking about relational anything for now. Use an object database if
you have to. This line of thinking goes with LDAP as well and hence is
a big factor in recommending the use of the Naming provider over the Eve
provider. These providers for configuration complenent each other at
different ends of the spectrum.
> I'm not yet familiar enough with Eve to say, but it would have to be
> able to perform all the functionality of naming without requiring an
> onerous amount of additional configuration or dependencies to make
> sense. I'm not sure this would be the case. What probably makes more
> sense to have Eve use naming if that makes any sense, and keep naming
> as a small library for the purposes of (2).
>
> However, being able to embed Eve offers richer functionality to an
> application that might be useful even for doing (2).
>
>>
>> FWIW, I would not want to see Tomcat or Geronimo completely abandon
>> 2) -- i.e., become wholly dependent on centralized "administrative
>> repositories". There are some practical advantages to having basic
>> configuration information available locally.
>>
Well again this comes with a move from being a standalone app server to
one that snaps in to an enterprise to enable configuration via
centralized interfaces. Imagine making changes to several instances of
Geronimo in a cluster. This is the sort of thing that Eve will provide
value for.
> I agree - one reason to use JNDI for the client is to be able to
> switch between or combine two depending on the deployment environment.
>
Well the switch in theory is nice but not possible between Eve and
Naming due to lack of namespace alignment. Eve is an LDAP namespace
provider while Naming is for java:enc. So you cannot interchange
between the two without considerable code changes. However going form
Naming to another java:enc provider should not be an issue. Likewise
switching between the Eve provider and the SUN JNDI LDAP provider should
not require code changes.
Alex
Re: Embedding Eve Was: To release or not to release?
Posted by Brett Porter <br...@apache.org>.
Phil Steitz wrote:
> From the various recent posts, it seems we have two basic use cases
> here and possibly two solutions for one of them -- maybe not a bad
> thing, but something to think about.
>
> 1) Centralized access to configuration information and object
> references for physically distributed systems
>
> 2) Bootstrapping and providing local, in-memory access to
> configuration / object references based on information in local flat
> files.
>
> In Java, we want to use the JNDI client API in both cases. 1) is a
> classic application for Directory Services and certainly part of the
> motivation for Eve. For 2) either embedded Eve or [naming] will work.
> I am interested in understanding the pros and cons of these two
> approaches. We may want to merge them in some way (i.e., modify
> [naming] to just front embedded Eve). Does this make sense?
I'm not yet familiar enough with Eve to say, but it would have to be
able to perform all the functionality of naming without requiring an
onerous amount of additional configuration or dependencies to make
sense. I'm not sure this would be the case. What probably makes more
sense to have Eve use naming if that makes any sense, and keep naming as
a small library for the purposes of (2).
However, being able to embed Eve offers richer functionality to an
application that might be useful even for doing (2).
>
> FWIW, I would not want to see Tomcat or Geronimo completely abandon 2)
> -- i.e., become wholly dependent on centralized "administrative
> repositories". There are some practical advantages to having basic
> configuration information available locally.
>
I agree - one reason to use JNDI for the client is to be able to switch
between or combine two depending on the deployment environment.
Cheers,
Brett
Re: Embedding Eve Was: To release or not to release?
Posted by Phil Steitz <ph...@steitz.com>.
Brett Porter wrote:
>>I want a JNDI server which gets its data from commons-config; and not just
>>use the JndiConfiguration part of commons-config (which I really need to
>>learn more about).
>
>
> Sorry, I remember you mentioning this in the past. I had it back-to-front.
>
> So would replacing XmlConfigurator in directory-naming with
> commons-configuration suite your needs?
I think that is a good idea. Having the input format flexibility of
commons-configuration would be a very good thing for [naming].
Seems like a good idea to me, as it
> could continue to read the current configuration files as well.
>
> Or do you want more than an in-memory provider, and need Eve itself to do this?
From the various recent posts, it seems we have two basic use cases here
and possibly two solutions for one of them -- maybe not a bad thing, but
something to think about.
1) Centralized access to configuration information and object references
for physically distributed systems
2) Bootstrapping and providing local, in-memory access to configuration /
object references based on information in local flat files.
In Java, we want to use the JNDI client API in both cases. 1) is a classic
application for Directory Services and certainly part of the motivation
for Eve. For 2) either embedded Eve or [naming] will work. I am
interested in understanding the pros and cons of these two approaches. We
may want to merge them in some way (i.e., modify [naming] to just front
embedded Eve). Does this make sense?
FWIW, I would not want to see Tomcat or Geronimo completely abandon 2) --
i.e., become wholly dependent on centralized "administrative
repositories". There are some practical advantages to having basic
configuration information available locally.
Phil
>
> - Brett
Re: Embedding Eve Was: To release or not to release?
Posted by Brett Porter <br...@apache.org>.
> I want a JNDI server which gets its data from commons-config; and not just
> use the JndiConfiguration part of commons-config (which I really need to
> learn more about).
Sorry, I remember you mentioning this in the past. I had it back-to-front.
So would replacing XmlConfigurator in directory-naming with
commons-configuration suite your needs? Seems like a good idea to me, as it
could continue to read the current configuration files as well.
Or do you want more than an in-memory provider, and need Eve itself to do this?
- Brett
Re: Embedding Eve Was: To release or not to release?
Posted by Martin Wegner <ma...@yahoo.com>.
Henri,
Have you considered using DSML files instead of .property files? DSML is
a much better fit since it really is the LDAP tree structure expressed in
XML. Having a DSML back end (or using the DSML JNDI Provider) would be a
slick solution.
--Marty
--- Henri Yandell <ba...@generationjava.com> wrote:
>
> Just to underline:
>
> I want a JNDI server which gets its data from commons-config; and not
> just
> use the JndiConfiguration part of commons-config (which I really need to
>
> learn more about).
>
> In effect, I want the Eve database to be based on a directory structure
> of
> .properties files. Simple-JNDI does this, but it supports very little of
>
> the JNDI API and I really don't want to reinvent the wheel.
>
> It might be a bit stupid as I'm basically going to have:
>
> Commons-config-as-database -> Eve/Directory Naming ->
> Jndi-Commons-Config
>
> Advantages being the ability to switch to LDAP over time; and the global
>
> nature of JNDI for configuration. IoC is okay in terms of passing a
> Config
> class around, but I see no reason not to still back that Config with
> JNDI.
>
> Hen
>
> On Wed, 24 Nov 2004, Brett Porter wrote:
>
> > Hi,
> >
> > I use directory naming for in memory JNDI + commons-configuration in
> > production, it works just fine :)
> >
> > Cheers,
> > Brett
> >
> >>
> >> Hope so, I want to kill simple-jndi, which is what I currently use as
> a way
> >> to embed a JNDI server for local configuration. Though, a big part is
> that
> >> it sits on top of static XML/property files.
> >>
> >> What kind of files would you make for Eve? My todo list has: attach
> >> commons-configuration to directory-naming, though I've no idea
> whether it's
> >> workable. Does Eve have this stuff already?
> >>
> >> Hen
> >>
> >>
> >>
> >
> >
> >
>
Re: Embedding Eve Was: To release or not to release?
Posted by Henri Yandell <ba...@generationjava.com>.
Just to underline:
I want a JNDI server which gets its data from commons-config; and not just
use the JndiConfiguration part of commons-config (which I really need to
learn more about).
In effect, I want the Eve database to be based on a directory structure of
.properties files. Simple-JNDI does this, but it supports very little of
the JNDI API and I really don't want to reinvent the wheel.
It might be a bit stupid as I'm basically going to have:
Commons-config-as-database -> Eve/Directory Naming -> Jndi-Commons-Config
Advantages being the ability to switch to LDAP over time; and the global
nature of JNDI for configuration. IoC is okay in terms of passing a Config
class around, but I see no reason not to still back that Config with JNDI.
Hen
On Wed, 24 Nov 2004, Brett Porter wrote:
> Hi,
>
> I use directory naming for in memory JNDI + commons-configuration in
> production, it works just fine :)
>
> Cheers,
> Brett
>
>>
>> Hope so, I want to kill simple-jndi, which is what I currently use as a way
>> to embed a JNDI server for local configuration. Though, a big part is that
>> it sits on top of static XML/property files.
>>
>> What kind of files would you make for Eve? My todo list has: attach
>> commons-configuration to directory-naming, though I've no idea whether it's
>> workable. Does Eve have this stuff already?
>>
>> Hen
>>
>>
>>
>
>
>
Re: Embedding Eve Was: To release or not to release?
Posted by Brett Porter <br...@apache.org>.
Hi,
I use directory naming for in memory JNDI + commons-configuration in
production, it works just fine :)
Cheers,
Brett
>
> Hope so, I want to kill simple-jndi, which is what I currently use as
> a way to embed a JNDI server for local configuration. Though, a big
> part is that it sits on top of static XML/property files.
>
> What kind of files would you make for Eve? My todo list has: attach
> commons-configuration to directory-naming, though I've no idea whether
> it's workable. Does Eve have this stuff already?
>
> Hen
>
>
>
Re: Embedding Eve Was: To release or not to release?
Posted by Alex Karasulu <ao...@bellsouth.net>.
Hey Henri,
Henri Yandell wrote:
>
>
> On Mon, 22 Nov 2004, Martin Wegner wrote:
>
>>> We definitely have embeddability in mind. The core stuff will be POJO
>>> and we'll provide, or help with, wrappers for well-known containers.
>>
>>
>> Great. What I hope happens is that once Eve is working and is
>> embeddable
>> that the Jarkarta teams will start to think about moving configuration
>> information from static XML/property files to Eve. By doing so we would
>
>
> Hope so, I want to kill simple-jndi, which is what I currently use as
> a way to embed a JNDI server for local configuration. Though, a big
> part is that it sits on top of static XML/property files.
>
> What kind of files would you make for Eve?
Eve has a modular backend subsystem where you plug in any kind of
database implementation that could forseeably use any kind of file
format for storing information. The default database that Eve's system
partition uses is based on JDBM which is a Java based implementation of
UNIX dbm. However note that the file formats are not the same between
JDBM and UNIX dbm. JDBM resembles Sleepycat's JE product more so than
UNIX dbm. It's basically an API for B+Trees on secondary storage.
Eve creates a set of these files for various indices and the master
table for any partition. So there will be several files within a folder
corresponding to a partition that hangs off of the namespace. For
example ou=system is for the system partition. Also note a file is
created for each user specified index as well. This is done basically
to keep the master db separate which can be used to regenerate all these
indices. It's one of two files you can use to rebuild the entire
database with. Most indices can be generated from it.
> My todo list has: attach commons-configuration to directory-naming,
> though I've no idea whether it's workable. Does Eve have this stuff
> already?
>
No but it's an excellent fit. Personally I would stick to the less
heavy weight flat files in commons configuration for small data sets.
When you're data grows beyond 10K and needs to be searchable this is
where the power of the BTree comes in handy.
Now this is for the default JDBM based database.
Enrique and I have been talking about building a Prevayler partition
which keeps all in memory. This might be an excellent lighter
alternative to the B+Tree based partition implementation. I really
think Prevalyer is worth looking at. Again this would be for moderate
to large data sets.
Alex
Re: Embedding Eve Was: To release or not to release?
Posted by Alex Karasulu <ao...@bellsouth.net>.
Martin Wegner wrote:
<snip/>
>>A classic place to start is see if the log4j config file can be moved into
>>an LDAP server. Imagine being able to edit the log4j config file using an
>>LDAP client instead of using a text editor. And then it can be shared
>>amongst multiple apps (if need be).
>>
>>
>>
Yeah that's one of the massive benefits of using a distributed database
like LDAP. Plus the hierarchical aspect works well with log4j's Logger
hierarchy as well (or was it Categories don't remember). Anyhow there
is a good fit here. Also Enrique in an email below made mention of
using the Java Preferences API. The Perferences API is also an ideal
candidate for being backed by a directory. Funny thing is I remember
reading somewhere in the Preferences API documentation about not having
to use heavy weight facilities like LDAP for storing this kind of data;
hence the reason for the Preferences API. I disagree with those remarks
though - they never knew about Eve as an embeddable lighter wight directory.
Anyhow this is a most excellent topic that only scratches the surface
for configuration. Really any peice of information that is...
o relatively static,
o needs to be read optimized,
o highly available for multiple systems to access
o and might be hierarchical in nature
is an ideal candidate for being backed by LDAP.
Another very unique fit for LDAP are resource bundles for distributed
applications. LDAP filters can use language tags to select attributes
specific to a partiticular language. For example I can have a label on
a webpage where the entry for the label has the following commonNames;
cn;lang-en: hello
cn;lang-fr; bonjour
cn;lang-de; hallo
cn;lang-es; hola
Just asking for cn may default to the language code associated with the
current locale or the locale of the user making the request. I could
explicitely ask for a specific version of the attribute as well such as
cn;lang-fr. Plus bundles can go the distance with internationalization
since LDAP strings are Unicode strings. There's definately lots of
synergy here but LDAP has been the child left behind for too long: most
people have just given up on it unfortunately.
>>The slippery slope of this suggestion is that no one has ever codified a
>>best practices in this area (server config in an LDAP server).
>>
True no one has done this for the gambit of J2EE application
configurations or component configurations out there. This is one of
the those things this project will have to address and address well.
IMO having the expertise of folks like Henri and Phil from the commons
will give us the vision we need to do that right. People like yourself
who see this vision are also critical.
>>So my
>>guess is that this group will stir up a number of hornets with this idea.
>>But that is a good sign.
>>
Oh I think so. Look M$ has leveraged the directory in .Net pretty well
behind the scences to help orchestrate component oriented systems. The
open source and Java world has lagged behind in this area for far too
long. Almost to a crippling degree. It's about time we leveraged this
core technology to help manage and orchestrate our distributed
components and systems. Otherwise with the geometric explosion of
components, users, systems, nodes and much more we're not going to fair
well in comparision to other technologies already leveraging the directory.
>>If we can get a set of best practices defined
>>(LDAP tree structure, DN construction, live update handling, etc.) then I
>>think the Eve server will have provided a benefit above and beyond the
>>LDAP server.
>>
>>
Great thoughts Marty. DIT structure and using LDAP properly is
something that is lacking en mass. It's pretty hard to make even
seasoned DBAs see how to contrain and manage a directory information
tree (DIT). The key lies in using Schema structures correctly. One of
our goals is to make LDAP in general easier to understand and hence
DIT's easier to design.
Take for example things like NameForms which define those attributes
that may be used for the RDN of an entry (structural objectclass) hence
effecting the possible DNs within your DIT namespace. Many people just
don't know about these things. I guess we are going to have to
predefine schemas for the configuration stuff to make sure they don't
have to grok these concepts to store simple configurations in their
directory. In the end though I don't want to dumb it down to the point
where things are not intuitive. Directory users should not loose power
for ease of use.
So yeah if we can overcome these hurdles gracefully we can really make a
big impact to the OS community and especially the Java community. These
and other reasons (i.e. lack of LDAP triggers) are the driving forces
behind the genesis of the Directory project.
Alex
Re: Embedding Eve Was: To release or not to release?
Posted by Enrique Rodriguez <er...@apache.org>.
Martin Wegner wrote:
>>What kind of files would you make for Eve? My todo list has: attach
>>commons-configuration to directory-naming, though I've no idea whether
>>it's workable. Does Eve have this stuff already?
>
> Commons-configuration is a good idea. I wrote my own implementation based
> on what I thought were limitations of commons-config but it is probably a
> better idea to start with something from Jakarta.
While you're on this topic, do you guys have any thoughts on where the
Preferences API fits into this picture? It's there in the JDK.
> A classic place to start is see if the log4j config file can be moved into
> an LDAP server. Imagine being able to edit the log4j config file using an
> LDAP client instead of using a text editor. And then it can be shared
> amongst multiple apps (if need be).
>
> The slippery slope of this suggestion is that no one has ever codified a
> best practices in this area (server config in an LDAP server). So my
> guess is that this group will stir up a number of hornets with this idea.
> But that is a good sign. If we can get a set of best practices defined
> (LDAP tree structure, DN construction, live update handling, etc.) then I
> think the Eve server will have provided a benefit above and beyond the
> LDAP server.
>
>
> --Marty
Re: Embedding Eve Was: To release or not to release?
Posted by Martin Wegner <ma...@yahoo.com>.
> What kind of files would you make for Eve? My todo list has: attach
> commons-configuration to directory-naming, though I've no idea whether
> it's workable. Does Eve have this stuff already?
Commons-configuration is a good idea. I wrote my own implementation based
on what I thought were limitations of commons-config but it is probably a
better idea to start with something from Jakarta.
A classic place to start is see if the log4j config file can be moved into
an LDAP server. Imagine being able to edit the log4j config file using an
LDAP client instead of using a text editor. And then it can be shared
amongst multiple apps (if need be).
The slippery slope of this suggestion is that no one has ever codified a
best practices in this area (server config in an LDAP server). So my
guess is that this group will stir up a number of hornets with this idea.
But that is a good sign. If we can get a set of best practices defined
(LDAP tree structure, DN construction, live update handling, etc.) then I
think the Eve server will have provided a benefit above and beyond the
LDAP server.
--Marty