You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Pete Robbins <ro...@googlemail.com> on 2006/11/30 07:50:46 UTC
[C++] Binding/Language Extensions - packaging
Our current method of packaging and loading an extension is fairly simple:
we load all schema and libraries in the extensions path. This has a number
of problems.
1. An extension may consist of more than one library e.g. libmy_extension.so
and libmy_extension_utils.so. Our current loading scheme will attempt to
load both of these and may fail on the one that doesn't provide the
extension initialize method. On MacOS the output of our build produces a
libx.dylib and a series of symlinks to this called libx.0.dylib,
libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't cause
problems as they are all the sam library and just repeatedly register to
handle the same requests. Very inefficient though!
2. Control of whether or not to load an extension library is currently by
renaming the library so the runtime doesn't find it. An example is that we
ship our python extension as libtuscany_sca_python.so.diabled. This is
horrible and error prone.
We could improve this by having a system configuration file that lists the
required extensions but the I like the self contained package approach that
we have now. I'd like to implement an improved scheme for packaging an
extension by introducing a per extension configuration file. Something like:
tuscany_sca_ws_binding.extension
<scacpp:extension name="ws binding" enabled="true">
<library name="tuscany_sca_ws_reference"/>
<library name="tuscany_sca_ws_service"/>
</scacppp:extension>
So the package would look like:
extensions/
ws/
tusany_sca_ws_binding.extension
lib/
xsd/
other_folder/
...
The .extension configuration file is saying to load the library which is
located somewhere in the package... the runtime will find it... no need to
specify a path.
Taking this further the configuration file could list the schema to be
loaded. Currently the runtime will just load any it finds but these may not
be needed by the runtime e.g. the schema may be for some extension
implementation specific purpose.
I think it would also be good for the extension initialization() method to
take as a parameter the root of the extension e.g.
extension("/tuscany/extensions/ws"). This would allow the extension package
to contain any configuration information that it needs.
I'd like to start by at least introducing the .extension file for each
extension and loading only the specified library(ies) if the extension is
enabled.
Any thoughts?
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
>
> <snip>
>
> > > I would package the cpp language extension as
> > >
> > > cpp/
> > > bin/
> > > lib/
> > > include/
> > > xsd/
> > > extension/
> > >
> > > The bin, lib, include are exactly what you would expect from a package
> that
> > > you might want to build against. The extension/ folder would hold the
> > > library that implements the extension interface and xsd holds... well
> xsds!
> > >
>
> How would extensions be disabled? Rename the extension dir? Remove any
> symlinks in extension/ ?
Remove symlink or the dll on windows
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Andrew Borley <aj...@gmail.com>.
<snip>
> > I would package the cpp language extension as
> >
> > cpp/
> > bin/
> > lib/
> > include/
> > xsd/
> > extension/
> >
> > The bin, lib, include are exactly what you would expect from a package that
> > you might want to build against. The extension/ folder would hold the
> > library that implements the extension interface and xsd holds... well xsds!
> >
How would extensions be disabled? Rename the extension dir? Remove any
symlinks in extension/ ?
Cheers
Andy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
>
>
>
> On 30/11/06, Jean-Sebastien Delfino <js...@apache.org> wrote:
> >
> > Simon Laws wrote:
> > > On 11/30/06, Pete Robbins <robbinspg@googlemail.com > wrote:
> > >>
> > >> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > >> >
> > >> > On 11/30/06, Pete Robbins < robbinspg@googlemail.com> wrote:
> > >> > >
> > >> > > On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
> > >> > > >
> > >> > > > On 11/30/06, Pete Robbins < robbinspg@googlemail.com> wrote:
> > >> > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > >> > > > > >
> > >> > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >> > > > > > >
> > >> > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com>
> > wrote:
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > On 30/11/06, Simon Laws <si...@googlemail.com>
> > >> wrote:
> > >> > > > > > > > >
> > >> > > > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com>
> > >> wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > On 30/11/06, Pete Robbins <robbinspg@googlemail.com
> > >
> > >> > wrote:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
> > >> > > jsdelfino@apache.org>
> > >> > > > > > > wrote:
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Pete Robbins wrote:
> > >> > > > > > > > > > > > > Our current method of packaging and loading
> > an
> > >> > > extension
> > >> > > > is
> > >> > > > > > > > > fairly
> > >> > > > > > > > > > > > > simple:
> > >> > > > > > > > > > > > > we load all schema and libraries in the
> > >> extensions
> > >> > > path.
> > >> > > > > > This
> > >> > > > > > > > > has a
> > >> > > > > > > > > > > > > number
> > >> > > > > > > > > > > > > of problems.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > 1. An extension may consist of more than one
> > >> library
> > >> > > e.g
> > >> > > > .
> > >> > > > > > > > > > > > > libmy_extension.so
> > >> > > > > > > > > > > > > and libmy_extension_utils.so. Our current
> > >> loading
> > >> > > scheme
> > >> > > > > > will
> > >> > > > > > > > > > attempt
> > >> > > > > > > > > > > > to
> > >> > > > > > > > > > > > > load both of these and may fail on the one
> > that
> > >> > > doesn't
> > >> > > > > > > provide
> > >> > > > > > > > > the
> > >> > > > > > > > > > > > > extension initialize method. On MacOS the
> > output
> > >> of
> > >> > > our
> > >> > > > > > build
> > >> > > > > > > > > > produces
> > >> > > > > > > > > > > > a
> > >> > > > > > > > > > > > > libx.dylib and a series of symlinks to this
> > >> called
> > >> > > > > > > libx.0.dylib,
> > >> > > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of
> > >> these
> > >> > > which
> > >> > > > > > > doesn't
> > >> > > > > > > > > > cause
> > >> > > > > > > > > > > > > problems as they are all the sam library and
> > >> just
> > >> > > > repeatedly
> > >> > > > > > > > > > register
> > >> > > > > > > > > > > > to
> > >> > > > > > > > > > > > > handle the same requests. Very inefficient
> > >> though!
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > 2. Control of whether or not to load an
> > >> extension
> > >> > > > library is
> > >> > > > > > > > > > currently
> > >> > > > > > > > > > > > by
> > >> > > > > > > > > > > > > renaming the library so the runtime doesn't
> > find
> > >> it.
> > >> > > An
> > >> > > > > > > example
> > >> > > > > > > > > is
> > >> > > > > > > > > > > > > that we
> > >> > > > > > > > > > > > > ship our python extension as
> > >> > > > > > libtuscany_sca_python.so.diabled.
> > >> > > > > > > > > This
> > >> > > > > > > > > > is
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > > horrible and error prone.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > We could improve this by having a system
> > >> > configuration
> > >> > > > file
> > >> > > > > > > that
> > >> > > > > > > > > > lists
> > >> > > > > > > > > > > > > the
> > >> > > > > > > > > > > > > required extensions but the I like the self
> > >> > contained
> > >> > > > > > package
> > >> > > > > > > > > > approach
> > >> > > > > > > > > > > > > that
> > >> > > > > > > > > > > > > we have now. I'd like to implement an
> > improved
> > >> > scheme
> > >> > > > for
> > >> > > > > > > > > packaging
> > >> > > > > > > > > > an
> > >> > > > > > > > > > > > > extension by introducing a per extension
> > >> > configuration
> > >> > > > file.
> > >> > > > > > > > > > Something
> > >> > > > > > > > > > > > > like:
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > tuscany_sca_ws_binding.extension
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > <scacpp:extension name="ws binding"
> > >> enabled="true">
> > >> > > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > >> > > > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > >> > > > > > > > > > > > > </scacppp:extension>
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > So the package would look like:
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > extensions/
> > >> > > > > > > > > > > > > ws/
> > >> > > > > > > > > > > > > tusany_sca_ws_binding.extension
> > >> > > > > > > > > > > > > lib/
> > >> > > > > > > > > > > > > xsd/
> > >> > > > > > > > > > > > > other_folder/
> > >> > > > > > > > > > > > > ...
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > The .extension configuration file is saying
> > to
> > >> load
> > >> > > the
> > >> > > > > > > library
> > >> > > > > > > > > > which
> > >> > > > > > > > > > > > is
> > >> > > > > > > > > > > > > located somewhere in the package... the
> > runtime
> > >> will
> > >> > > > find
> > >> > > > > > > it...
> > >> > > > > > > > > no
> > >> > > > > > > > > > > > > need to
> > >> > > > > > > > > > > > > specify a path.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > Taking this further the configuration file
> > could
> > >> > list
> > >> > > > the
> > >> > > > > > > schema
> > >> > > > > > > > > to
> > >> > > > > > > > > > be
> > >> > > > > > > > > > > > > loaded. Currently the runtime will just load
> > any
> > >> it
> > >> > > > finds
> > >> > > > > > but
> > >> > > > > > > > > these
> > >> > > > > > > > > > > > > may not
> > >> > > > > > > > > > > > > be needed by the runtime e.g. the schema may
> > be
> > >> for
> > >> > > some
> > >> > > > > > > > > extension
> > >> > > > > > > > > > > > > implementation specific purpose.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > I think it would also be good for the
> > extension
> > >> > > > > > > initialization()
> > >> > > > > > > > > > > > > method to
> > >> > > > > > > > > > > > > take as a parameter the root of the extension
> >
> > >> e.g.
> > >> > > > > > > > > > > > > extension("/tuscany/extensions/ws"). This
> > would
> > >> > allow
> > >> > > > the
> > >> > > > > > > > > extension
> > >> > > > > > > > > > > > > package
> > >> > > > > > > > > > > > > to contain any configuration information that
> > it
> > >> > > needs.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > I'd like to start by at least introducing the
> > >> > > .extension
> > >> > > > > > file
> > >> > > > > > > > > for
> > >> > > > > > > > > > each
> > >> > > > > > > > > > > > > extension and loading only the specified
> > >> > library(ies)
> > >> > > if
> > >> > > > the
> > >> > > > > > > > > > extension
> > >> > > > > > > > > > > > is
> > >> > > > > > > > > > > > > enabled.
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > > Any thoughts?
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > > >
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Two thoughts:
> > >> > > > > > > > > > > > - convention over configuration
> > >> > > > > > > > > > > > - the runtime should be consumable without
> > >> having to
> > >> > go
> > >> > > > tweak
> > >> > > > > > > XML
> > >> > > > > > > > > > > > configuration files
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > If I remember correctly, renaming the dlls to
> > >> > .disabled
> > >> > > > was a
> > >> > > > > > > last
> > >> > > > > > > > >
> > >> > > > > > > > > > > > minute change to work around DLL loading errors
> > >> with
> > >> > our
> > >> > > > M2
> > >> > > > > > > > > release on
> > >> > > > > > > > > > > > Windows.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > I agree that we should do better than renaming
> > to
> > >> > > > .disabled,
> > >> > > > > > but
> > >> > > > > > > > > I'd
> > >> > > > > > > > > > > > like to understand better the actual issues
> > >> that we
> > >> > > faced
> > >> > > > > > before
> > >> > > > > > > > > > > > inventing yet another XML based
> > >> runtime configuration
> > >> > > > > > language
> > >> > > > > > > :)
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > I'm aware of the following issues:
> > >> > > > > > > > > > > > 1. We need the runtime to load extension libs
> > >> only,
> > >> > not
> > >> > > > other
> > >> > > > > > > libs
> > >> > > > > > > > > > under
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > the extension directory which are not actual
> > >> > extensions
> > >> > > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that
> > >> contribute
> > >> > > to
> > >> > > > > > SCDL,
> > >> > > > > > > > > and
> > >> > > > > > > > > > > > leave other XSDs under the extension directory
> > >> alone
> > >> > > > > > > > > > > > 3. Extensions that cannot be loaded because
> > >> some of
> > >> > > their
> > >> > > > > > > > > dependencies
> > >> > > > > > > > > > > > are not there should no break the runtime
> > >> > > > > > > > > > > > 4. The system admin / installer should be able
> > to
> > >> > > disable
> > >> > > > > > > > > extensions
> > >> > > > > > > > > > > > that won't load because their dependencies are
> > not
> > >> > there
> > >> > > > (I'm
> > >> > > > > > > not
> > >> > > > > > > > > yet
> > >> > > > > > > > > > > > convinced that this is still an issue if we
> > manage
> > >> to
> > >> > > > solve
> > >> > > > > > > issue
> > >> > > > > > > > > #3)
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Did we run into any other big issues?
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > No, that's about it.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > 2. XSD loading can be done by convention (schemas
> >
> > >> for
> > >> > the
> > >> > > > > > runtime
> > >> > > > > > > > > are
> > >> > > > > > > > > > > always in a folder called 'xsd'
> > >> > > > > > > > > > > 1. Could be solved in a similar way by only
> > loading
> > >> > > > libraries in
> > >> > > > > > a
> > >> > > > > > > > > > folder
> > >> > > > > > > > > > > called ???
> > >> > > > > > > > > > > 3. Can be solved by just ignoring the load
> > >> > errors/issuing
> > >> > > a
> > >> > > > > > > warning
> > >> > > > > > > > > > rather
> > >> > > > > > > > > > > than giving up
> > >> > > > > > > > > > > 4. Can be solved by solution to 3.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > I recall now why 3. was a big problem. Windows
> > >> sometimes
> > >> > > > throws up
> > >> > > > > > > an
> > >> > > > > > > > > > error
> > >> > > > > > > > > > dialog when the load fails so it was not just a
> > >> case of
> > >> > the
> > >> > > > > > runtime
> > >> > > > > > > > > > handling
> > >> > > > > > > > > > the load failure so we had to "disable" the
> > extension.
> > >> > > > > > > > > >
> > >> > > > > > > > > > Cheers,
> > >> > > > > > > > > >
> > >> > > > > > > > > > --
> > >> > > > > > > > > > Pete
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > --
> > >> > > > > > > > > > Pete
> > >> > > > > > > > > >
> > >> > > > > > > > > > Pete, just summarizing so that I understand. It
> > seems
> > >> > there
> > >> > > > are
> > >> > > > > > two
> > >> > > > > > > > > parts
> > >> > > > > > > > > to this.
> > >> > > > > > > > >
> > >> > > > > > > > > A/ Building and installing an extension in order that
> > it
> > >> can
> > >> > > be
> > >> > > > > > > consumed
> > >> > > > > > > > > by
> > >> > > > > > > > > the runtime. Building and installing extensions is
> > >> > potentially
> > >> > > a
> > >> > > > > > > > > separate
> > >> > > > > > > > > activity from building and installing the runtime
> > >> itself.
> > >> > > > Looking at
> > >> > > > > > > the
> > >> > > > > > > > > M2
> > >> > > > > > > > > download they all come together at the moment.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Yes. It just happens that we are the only providers of
> > >> > > extensions
> > >> > > > at
> > >> > > > > > the
> > >> > > > > > > > moment.
> > >> > > > > > > >
> > >> > > > > > > > B/ Optionally consuming/enabling an extension in the
> > >> runtime,
> > >> > > once
> > >> > > > the
> > >> > > > > > > > > extension has been installed, in such a way that the
> > >> runtime
> > >> > > is
> > >> > > > able
> > >> > > > > > > to
> > >> > > > > > > > > find
> > >> > > > > > > > > and load it.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Yes.
> > >> > > > > > > >
> > >> > > > > > > > If I understand the previous posts (and the current
> > >> code) A/
> > >> > is
> > >> > > > > > achieved
> > >> > > > > > > > > by
> > >> > > > > > > > > building the extension in the deployed folder layout
> > >> > > > > > > > > extension type/
> > >> > > > > > > > > bin
> > >> > > > > > > > > lib
> > >> > > > > > > > > xsd
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > The current runtime does not mandate any layout. It
> > will
> > >> seek
> > >> > > and
> > >> > > > > > > > destroy.. I mean seek and load any library it finds and
> > >> any
> > >> > .xsd
> > >> > > > it
> > >> > > > > > > finds.
> > >> > > > > > > > That layout is just how we package extensions in ou
> > >> build at
> > >> > the
> > >> > > > > > moment.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > and Pete you are proposing that this is adjusted so
> > that
> > >> there
> > >> > > are
> > >> > > > > > > > > directories that hold just the stuff that the
> > >> contains the
> > >> > > > extension
> > >> > > > > > > > > rather
> > >> > > > > > > > > than the things it depends on.
> > >> > > > > > > > > extension type/
> > >> > > > > > > > > bin
> > >> > > > > > > > > extensionbin?
> > >> > > > > > > > > lib
> > >> > > > > > > > > xsd
> > >> > > > > > > > > extensionxsd?
> > >> > > > > > > > >
> > >> > > > > > > > > You are also proposing that B/ is achieved by
> > ensuring
> > >> that
> > >> > > this
> > >> > > > > > > > > directory
> > >> > > > > > > > > structure be placed in a location that the the
> > >> runtime can
> > >> > > > search
> > >> > > > > > for
> > >> > > > > > > > > active
> > >> > > > > > > > > extensions (as it does at the moment in the deploy
> > >> > directory)
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > it currently loads any extensions from under the
> > >> > > > > > > > <sca_install_dir>/extensions folder
> > >> > > > > > > >
> > >> > > > > > > > and that the
> > >> > > > > > > > > runtime ignores any badly configured extensions if
> > >> possible.
> > >> > > > > > Removing
> > >> > > > > > > an
> > >> > > > > > > > > extension's directory from the deploy
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > extensions not deploy
> > >> > > > > > > >
> > >> > > > > > > > directory structure has the effect of
> > >> > > > > > > > > disabling it so I guess this is the fallback if the
> > >> runtime
> > >> > > > can't
> > >> > > > > > > > > continue.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > That is true in today's implementation.
> > >> > > > > > > >
> > >> > > > > > > > Anything more complex and, as Jean-Sebastien suggests,
> > you
> > >> > start
> > >> > > > > > getting
> > >> > > > > > > > > into the full blown package and dependency management
> >
> > >> > problem
> > >> > > > that
> > >> > > > > > > many
> > >> > > > > > > > > other systems try to solve in different ways, e.g.
> > >> rpm or
> > >> > > pear.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Agreed... I'll give up on that plan ;-)
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > What we need is to enable the runtime to identify which
> > >> > > extension
> > >> > > > > > > > libraries to load and which schema to load. My
> > suggestion
> > >> is:
> > >> > > > > > > >
> > >> > > > > > > > my_extension/
> > >> > > > > > > > bin/
> > >> > > > > > > > extension/
> > >> > > > > > > > library_that_will_be_loaded_by_the_runtime.so
> > >> > > > > > > > lib/
> > >> > > > > > > > lib_my_extension.so
> > >> > > > > > > > include/
> > >> > > > > > > > ... some headers maybe
> > >> > > > > > > > xsd/
> > >> > > > > > > > my_binding.xsd
> > >> > > > > > > > any_other_folder/
> > >> > > > > > > > any_stuff_I_like.xsd
> > >> > > > > > > >
> > >> > > > > > > > So the runtime will load any xsd that is in a folder
> > named
> > >> xsd
> > >> > > and
> > >> > > > > > > attempt
> > >> > > > > > > > to load any library in a folder called extension. It
> > would
> > >> not
> > >> > > > attempt
> > >> > > > > > > to
> > >> > > > > > > > load the any_stuff_I_like.xsd or the
> > lib_my_extension.so
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Cheers,
> > >> > > > > > > >
> > >> > > > > > > > --
> > >> > > > > > > > Pete
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > --
> > >> > > > > > > Pete
> > >> > > > > > >
> > >> > > > > > > Sounds like the right idea to me. What goes in the bin
> > dir?.
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > Anything you like!
> > >> > > >
> > >> > > >
> > >> > > > On windows, .exe's and .dll's are often found in bin, whereas
> > the
> > >> > > > .lib's (used for linking against at compile time) are generally
> >
> > >> found
> > >> > > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib
> > >> (which
> > >> > > > is more unix-like, and, IMO, nicer)
> > >> > > >
> > >> > > > >
> > >> > > > > Should the
> > >> > > > > > lib be...
> > >> > > > > >
> > >> > > > > > lib/
> > >> > > > > > lib_my_extension.lib
> > >> > > > >
> > >> > > > >
> > >> > > > > lif you like... again... anything you like.
> > >> > > > >
> > >> > > > > Maybe you could stick with bin,include, lib,xsd and then have
> >
> > >> your
> > >> > > > "other"
> > >> > > > > > directory have everything else in it that the extension
> > might
> > >> rely
> > >> > > on.
> > >> > > > > > Depends what you now anticipate being in bin and lib over
> > and
> > >> > above
> > >> > > > the
> > >> > > > > > actual library exposing the extension.
> > >> > > > > >
> > >> > > > > > Will you continue to mandate a specific place where
> > extensions
> > >> are
> > >> > > > > > installed, i.e. from you post <sca_install_dir>/extensions
> > >> folder?
> > >> > > > >
> > >> > > > >
> > >> > > > > Yes
> > >> > > > >
> > >> > > > > So that
> > >> > > > > > you can move an extension out of there if you don't want it
> > to
> > >> > load.
> > >> > > > >
> > >> > > > >
> > >> > > > > Well the idea here is you don't need to remove the whole
> > >> > package..just
> > >> > > > the
> > >> > > > > library from the extension dir
> > >> > > > >
> > >> > > > > Simon
> > >> > > > > >
> > >> > > > > >
> > >> > > > > The layout of the extension package is whatever the package
> > >> creater
> > >> > > > wants to
> > >> > > > > make it. We don't care. All we will care about is loading any
> > >> > library
> > >> > > in
> > >> > > > a
> > >> > > > > folder anywhere in the extension package that is in a folder
> > >> called
> > >> > > > > 'extension' and expecting it to implement the extension
> > >> initialize
> > >> > > > method.
> > >> > > > > We will also load any schema files in any folder called
> > >> 'xsd'. An
> > >> > > > extension
> > >> > > > > only needs the xsd folder if it has schema required for the
> > >> runtime
> > >> > > > model.
> > >> > > > >
> > >> > > > > As an example here is the ws binding extension implemented
> > for
> > >> > axis2c:
> > >> > > > >
> > >> > > > > <sca_install>/extensions/
> > >> > > > > ... somewhere in this tree... could be anywhere...
> > >> > > > > ws/
> > >> > > > > xsd/
> > >> > > > > ws-binding.xsd
> > >> > > > > reference/
> > >> > > > > lib/
> > >> > > > > libtuscany_sca_reference.so
> > >> > > > > service/
> > >> > > > > lib/
> > >> > > > > libtuscany_sca_service.so
> > >> > > > >
> > >> > > > >
> > >> > > > > OK... today the runtime would load the ws-binding.xsd as it
> > >> loads
> > >> > any
> > >> > > > xsd it
> > >> > > > > finds. It would also load any .so it finds so both reference
> > >> aand
> > >> > > > service
> > >> > > > > extensions would be loaded/initialized.
> > >> > > > >
> > >> > > > > With my proposed change the xsd would be loaded (as it's in a
> > >> folder
> > >> > > > named
> > >> > > > > 'xsd' ) but the libraries would not. To get the extensions
> > >> loaded
> > >> > you
> > >> > > > would
> > >> > > > > have to package with an 'extension' folder ... again anywhere
> > in
> > >> the
> > >> > > > tree...
> > >> > > > > which "contains" the libraries to load eg.
> > >> > > > >
> > >> > > > > ws/
> > >> > > > > extension/
> > >> > > > > libload_this.so ->
> > >> reference/lib/libtuscany_sca_reference.so
> > >> > > > > xsd/
> > >> > > > > ws-binding.xsd
> > >> > > > > reference/
> > >> > > > > lib/
> > >> > > > > libtuscany_sca_reference.so
> > >> > > > > service/
> > >> > > > > lib/
> > >> > > > > libtuscany_sca_service.so
> > >> > > > >
> > >> > > > > Here the runtime loads anything in any folder named
> > >> 'extension' so
> > >> > the
> > >> > > > > reference extension would be loaded/initialized.
> > >> > > >
> > >> > > > Windows doesn't really do symlinks - I guess on windows we'd
> > have
> > >> the
> > >> > > > tuscany_sca_reference.dll in extension/ and nothing in
> > >> reference/lib/
> > >> > > > or reference/bin/? For extensions that have dependencies on
> > other
> > >> > > > libraries, those dll's could go in lib/
> > >> > >
> > >> > >
> > >> > > I would actually have a copy of the dll in extension on windows
> > and
> > >> > delete
> > >> > > it to disable. For a large extension it would be very wasy to
> > make a
> > >> > tiny
> > >> > > dll which just implements the initialise interface expected by
> > the
> > >> > runtime
> > >> > > and leave everything else in a separate dll in bin/lib.
> > >> > >
> > >> > > > I would package the cpp language extension as
> > >> > > > >
> > >> > > > > cpp/
> > >> > > > > bin/
> > >> > > > > lib/
> > >> > > > > include/
> > >> > > > > xsd/
> > >> > > > > extension/
> > >> > > > >
> > >> > > > > The bin, lib, include are exactly what you would expect from
> > a
> > >> > package
> > >> > > > that
> > >> > > > > you might want to build against. The extension/ folder would
> > >> hold
> > >> > the
> > >> > > > > library that implements the extension interface and xsd
> > holds...
> > >> > well
> > >> > > > xsds!
> > >> > > > >
> > >> > > >
> > >> > > > All sounds good to me. +1
> > >> > > >
> > >> > > > Andy
> > >> > > >
> > >> > > >
> > >> ---------------------------------------------------------------------
> > >> > > > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > >> > > > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Pete
> > >> > >
> > >> > > Ok, so I think I get it. My concern is that giving the developer
> > >> the
> > >> > flexibility to put "extension" and "xsd" anywhere they like means
> > that
> > >> > they
> > >> > will accidentally duplicate these directories as these are pretty
> > >> common
> > >> > names for directories. So we could ask them always to have them at
> > the
> > >> top
> > >> > level. Doesn't remove the possibility for duplication but reduces
> > the
> > >> risk
> > >> > a
> > >> > little. Or we could make the names a little more exotic with some
> > >> > qualification "sca_extension", "sca_xsd" for example, and maintain
> > the
> > >> > flexibility you want.
> > >> >
> > >> > Simon
> > >> >
> > >> >
> > >>
> > >> Yes... the names are always the hardest thing to come up with! They
> > >> should
> > >> not be sca_ as this is Tuscany specific.
> > >>
> > >> so... we could get away with just the one folder tuscany_extension
> > that
> > >> contains the schema and libraries:
> > >>
> > >>
> > >> cpp/
> > >> bin/
> > >> lib/
> > >> include/
> > >> xsd/
> > >> tuscany_extension/ <- schema and libraries implementing extension
> > i/f
> > >> live under this tree
> > >> xsd -> ../xsd <- could even be a symlink to where they really
> > are
> > >>
> > >> I like that better. Runtime will load library and schema from any
> > >> subtree
> > >> under <sca_install>/extensions that is named 'tuscany_extension' and
> > >> ignore
> > >> anything else.
> > >>
> > >> Nice and flexible.
> > >>
> > >>
> > >> --
> > >> Pete
> > >>
> > >> That sounds like a plan. Nice one.
> > >
> > > Simon
> > >
> >
> > Wow, 15 messages on this thread since last evening :) The latest
> > proposal looks really good to me, nice, flexible and simple!
> >
> > I would only propose to shorten "tuscany_extension" to something like
> > "module" as we are already under an "extensions" directory maybe? This
> > terminology would be similar to Apache httpd and Axis2C as well, but
> > this is really nitpicking so if you prefer "tuscany_extension" I'm
> > perfectly fine with it too. I could even convince myself that I actually
> >
> > prefer "tuscany_extension" :) I just wanted to mention it to see what
> > you guys think...
>
>
> as I said, the naming is the hardest part!!!
>
> tuscany_extension is too long. I would have gone with just extension but I
> think module is the best so far.
>
> Cheers,
>
>
> --
> Pete
>
This is the proposal I intend to implement as per my note on the M3 status
thread.
Cheers,
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Jean-Sebastien Delfino <js...@apache.org> wrote:
>
> Simon Laws wrote:
> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >>
> >> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> >> >
> >> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >> > >
> >> > > On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
> >> > > >
> >> > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >> > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> >> > > > > >
> >> > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >> > > > > > >
> >> > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On 30/11/06, Simon Laws <si...@googlemail.com>
> >> wrote:
> >> > > > > > > > >
> >> > > > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com>
> >> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com>
> >> > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
> >> > > jsdelfino@apache.org>
> >> > > > > > > wrote:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Pete Robbins wrote:
> >> > > > > > > > > > > > > Our current method of packaging and loading an
> >> > > extension
> >> > > > is
> >> > > > > > > > > fairly
> >> > > > > > > > > > > > > simple:
> >> > > > > > > > > > > > > we load all schema and libraries in the
> >> extensions
> >> > > path.
> >> > > > > > This
> >> > > > > > > > > has a
> >> > > > > > > > > > > > > number
> >> > > > > > > > > > > > > of problems.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > 1. An extension may consist of more than one
> >> library
> >> > > e.g
> >> > > > .
> >> > > > > > > > > > > > > libmy_extension.so
> >> > > > > > > > > > > > > and libmy_extension_utils.so. Our current
> >> loading
> >> > > scheme
> >> > > > > > will
> >> > > > > > > > > > attempt
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > load both of these and may fail on the one that
> >> > > doesn't
> >> > > > > > > provide
> >> > > > > > > > > the
> >> > > > > > > > > > > > > extension initialize method. On MacOS the
> output
> >> of
> >> > > our
> >> > > > > > build
> >> > > > > > > > > > produces
> >> > > > > > > > > > > > a
> >> > > > > > > > > > > > > libx.dylib and a series of symlinks to this
> >> called
> >> > > > > > > libx.0.dylib,
> >> > > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of
> >> these
> >> > > which
> >> > > > > > > doesn't
> >> > > > > > > > > > cause
> >> > > > > > > > > > > > > problems as they are all the sam library and
> >> just
> >> > > > repeatedly
> >> > > > > > > > > > register
> >> > > > > > > > > > > > to
> >> > > > > > > > > > > > > handle the same requests. Very inefficient
> >> though!
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > 2. Control of whether or not to load an
> >> extension
> >> > > > library is
> >> > > > > > > > > > currently
> >> > > > > > > > > > > > by
> >> > > > > > > > > > > > > renaming the library so the runtime doesn't
> find
> >> it.
> >> > > An
> >> > > > > > > example
> >> > > > > > > > > is
> >> > > > > > > > > > > > > that we
> >> > > > > > > > > > > > > ship our python extension as
> >> > > > > > libtuscany_sca_python.so.diabled.
> >> > > > > > > > > This
> >> > > > > > > > > > is
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > > horrible and error prone.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > We could improve this by having a system
> >> > configuration
> >> > > > file
> >> > > > > > > that
> >> > > > > > > > > > lists
> >> > > > > > > > > > > > > the
> >> > > > > > > > > > > > > required extensions but the I like the self
> >> > contained
> >> > > > > > package
> >> > > > > > > > > > approach
> >> > > > > > > > > > > > > that
> >> > > > > > > > > > > > > we have now. I'd like to implement an improved
> >> > scheme
> >> > > > for
> >> > > > > > > > > packaging
> >> > > > > > > > > > an
> >> > > > > > > > > > > > > extension by introducing a per extension
> >> > configuration
> >> > > > file.
> >> > > > > > > > > > Something
> >> > > > > > > > > > > > > like:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > tuscany_sca_ws_binding.extension
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > <scacpp:extension name="ws binding"
> >> enabled="true">
> >> > > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> >> > > > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> >> > > > > > > > > > > > > </scacppp:extension>
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > So the package would look like:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > extensions/
> >> > > > > > > > > > > > > ws/
> >> > > > > > > > > > > > > tusany_sca_ws_binding.extension
> >> > > > > > > > > > > > > lib/
> >> > > > > > > > > > > > > xsd/
> >> > > > > > > > > > > > > other_folder/
> >> > > > > > > > > > > > > ...
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > The .extension configuration file is saying to
> >> load
> >> > > the
> >> > > > > > > library
> >> > > > > > > > > > which
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > located somewhere in the package... the runtime
> >> will
> >> > > > find
> >> > > > > > > it...
> >> > > > > > > > > no
> >> > > > > > > > > > > > > need to
> >> > > > > > > > > > > > > specify a path.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Taking this further the configuration file
> could
> >> > list
> >> > > > the
> >> > > > > > > schema
> >> > > > > > > > > to
> >> > > > > > > > > > be
> >> > > > > > > > > > > > > loaded. Currently the runtime will just load
> any
> >> it
> >> > > > finds
> >> > > > > > but
> >> > > > > > > > > these
> >> > > > > > > > > > > > > may not
> >> > > > > > > > > > > > > be needed by the runtime e.g. the schema may be
> >> for
> >> > > some
> >> > > > > > > > > extension
> >> > > > > > > > > > > > > implementation specific purpose.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > I think it would also be good for the extension
> >> > > > > > > initialization()
> >> > > > > > > > > > > > > method to
> >> > > > > > > > > > > > > take as a parameter the root of the extension
> >> e.g.
> >> > > > > > > > > > > > > extension("/tuscany/extensions/ws"). This would
> >> > allow
> >> > > > the
> >> > > > > > > > > extension
> >> > > > > > > > > > > > > package
> >> > > > > > > > > > > > > to contain any configuration information that
> it
> >> > > needs.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > I'd like to start by at least introducing the
> >> > > .extension
> >> > > > > > file
> >> > > > > > > > > for
> >> > > > > > > > > > each
> >> > > > > > > > > > > > > extension and loading only the specified
> >> > library(ies)
> >> > > if
> >> > > > the
> >> > > > > > > > > > extension
> >> > > > > > > > > > > > is
> >> > > > > > > > > > > > > enabled.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Any thoughts?
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Two thoughts:
> >> > > > > > > > > > > > - convention over configuration
> >> > > > > > > > > > > > - the runtime should be consumable without
> >> having to
> >> > go
> >> > > > tweak
> >> > > > > > > XML
> >> > > > > > > > > > > > configuration files
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > If I remember correctly, renaming the dlls to
> >> > .disabled
> >> > > > was a
> >> > > > > > > last
> >> > > > > > > > >
> >> > > > > > > > > > > > minute change to work around DLL loading errors
> >> with
> >> > our
> >> > > > M2
> >> > > > > > > > > release on
> >> > > > > > > > > > > > Windows.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > I agree that we should do better than renaming to
> >> > > > .disabled,
> >> > > > > > but
> >> > > > > > > > > I'd
> >> > > > > > > > > > > > like to understand better the actual issues
> >> that we
> >> > > faced
> >> > > > > > before
> >> > > > > > > > > > > > inventing yet another XML based
> >> runtime configuration
> >> > > > > > language
> >> > > > > > > :)
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > I'm aware of the following issues:
> >> > > > > > > > > > > > 1. We need the runtime to load extension libs
> >> only,
> >> > not
> >> > > > other
> >> > > > > > > libs
> >> > > > > > > > > > under
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > the extension directory which are not actual
> >> > extensions
> >> > > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that
> >> contribute
> >> > > to
> >> > > > > > SCDL,
> >> > > > > > > > > and
> >> > > > > > > > > > > > leave other XSDs under the extension directory
> >> alone
> >> > > > > > > > > > > > 3. Extensions that cannot be loaded because
> >> some of
> >> > > their
> >> > > > > > > > > dependencies
> >> > > > > > > > > > > > are not there should no break the runtime
> >> > > > > > > > > > > > 4. The system admin / installer should be able to
> >> > > disable
> >> > > > > > > > > extensions
> >> > > > > > > > > > > > that won't load because their dependencies are
> not
> >> > there
> >> > > > (I'm
> >> > > > > > > not
> >> > > > > > > > > yet
> >> > > > > > > > > > > > convinced that this is still an issue if we
> manage
> >> to
> >> > > > solve
> >> > > > > > > issue
> >> > > > > > > > > #3)
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Did we run into any other big issues?
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > No, that's about it.
> >> > > > > > > > > > >
> >> > > > > > > > > > > 2. XSD loading can be done by convention (schemas
> >> for
> >> > the
> >> > > > > > runtime
> >> > > > > > > > > are
> >> > > > > > > > > > > always in a folder called 'xsd'
> >> > > > > > > > > > > 1. Could be solved in a similar way by only loading
> >> > > > libraries in
> >> > > > > > a
> >> > > > > > > > > > folder
> >> > > > > > > > > > > called ???
> >> > > > > > > > > > > 3. Can be solved by just ignoring the load
> >> > errors/issuing
> >> > > a
> >> > > > > > > warning
> >> > > > > > > > > > rather
> >> > > > > > > > > > > than giving up
> >> > > > > > > > > > > 4. Can be solved by solution to 3.
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > I recall now why 3. was a big problem. Windows
> >> sometimes
> >> > > > throws up
> >> > > > > > > an
> >> > > > > > > > > > error
> >> > > > > > > > > > dialog when the load fails so it was not just a
> >> case of
> >> > the
> >> > > > > > runtime
> >> > > > > > > > > > handling
> >> > > > > > > > > > the load failure so we had to "disable" the
> extension.
> >> > > > > > > > > >
> >> > > > > > > > > > Cheers,
> >> > > > > > > > > >
> >> > > > > > > > > > --
> >> > > > > > > > > > Pete
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > --
> >> > > > > > > > > > Pete
> >> > > > > > > > > >
> >> > > > > > > > > > Pete, just summarizing so that I understand. It seems
> >> > there
> >> > > > are
> >> > > > > > two
> >> > > > > > > > > parts
> >> > > > > > > > > to this.
> >> > > > > > > > >
> >> > > > > > > > > A/ Building and installing an extension in order that
> it
> >> can
> >> > > be
> >> > > > > > > consumed
> >> > > > > > > > > by
> >> > > > > > > > > the runtime. Building and installing extensions is
> >> > potentially
> >> > > a
> >> > > > > > > > > separate
> >> > > > > > > > > activity from building and installing the runtime
> >> itself.
> >> > > > Looking at
> >> > > > > > > the
> >> > > > > > > > > M2
> >> > > > > > > > > download they all come together at the moment.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Yes. It just happens that we are the only providers of
> >> > > extensions
> >> > > > at
> >> > > > > > the
> >> > > > > > > > moment.
> >> > > > > > > >
> >> > > > > > > > B/ Optionally consuming/enabling an extension in the
> >> runtime,
> >> > > once
> >> > > > the
> >> > > > > > > > > extension has been installed, in such a way that the
> >> runtime
> >> > > is
> >> > > > able
> >> > > > > > > to
> >> > > > > > > > > find
> >> > > > > > > > > and load it.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Yes.
> >> > > > > > > >
> >> > > > > > > > If I understand the previous posts (and the current
> >> code) A/
> >> > is
> >> > > > > > achieved
> >> > > > > > > > > by
> >> > > > > > > > > building the extension in the deployed folder layout
> >> > > > > > > > > extension type/
> >> > > > > > > > > bin
> >> > > > > > > > > lib
> >> > > > > > > > > xsd
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > The current runtime does not mandate any layout. It will
> >> seek
> >> > > and
> >> > > > > > > > destroy.. I mean seek and load any library it finds and
> >> any
> >> > .xsd
> >> > > > it
> >> > > > > > > finds.
> >> > > > > > > > That layout is just how we package extensions in ou
> >> build at
> >> > the
> >> > > > > > moment.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > and Pete you are proposing that this is adjusted so that
> >> there
> >> > > are
> >> > > > > > > > > directories that hold just the stuff that the
> >> contains the
> >> > > > extension
> >> > > > > > > > > rather
> >> > > > > > > > > than the things it depends on.
> >> > > > > > > > > extension type/
> >> > > > > > > > > bin
> >> > > > > > > > > extensionbin?
> >> > > > > > > > > lib
> >> > > > > > > > > xsd
> >> > > > > > > > > extensionxsd?
> >> > > > > > > > >
> >> > > > > > > > > You are also proposing that B/ is achieved by ensuring
> >> that
> >> > > this
> >> > > > > > > > > directory
> >> > > > > > > > > structure be placed in a location that the the
> >> runtime can
> >> > > > search
> >> > > > > > for
> >> > > > > > > > > active
> >> > > > > > > > > extensions (as it does at the moment in the deploy
> >> > directory)
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > it currently loads any extensions from under the
> >> > > > > > > > <sca_install_dir>/extensions folder
> >> > > > > > > >
> >> > > > > > > > and that the
> >> > > > > > > > > runtime ignores any badly configured extensions if
> >> possible.
> >> > > > > > Removing
> >> > > > > > > an
> >> > > > > > > > > extension's directory from the deploy
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > extensions not deploy
> >> > > > > > > >
> >> > > > > > > > directory structure has the effect of
> >> > > > > > > > > disabling it so I guess this is the fallback if the
> >> runtime
> >> > > > can't
> >> > > > > > > > > continue.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > That is true in today's implementation.
> >> > > > > > > >
> >> > > > > > > > Anything more complex and, as Jean-Sebastien suggests,
> you
> >> > start
> >> > > > > > getting
> >> > > > > > > > > into the full blown package and dependency management
> >> > problem
> >> > > > that
> >> > > > > > > many
> >> > > > > > > > > other systems try to solve in different ways, e.g.
> >> rpm or
> >> > > pear.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Agreed... I'll give up on that plan ;-)
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > What we need is to enable the runtime to identify which
> >> > > extension
> >> > > > > > > > libraries to load and which schema to load. My suggestion
> >> is:
> >> > > > > > > >
> >> > > > > > > > my_extension/
> >> > > > > > > > bin/
> >> > > > > > > > extension/
> >> > > > > > > > library_that_will_be_loaded_by_the_runtime.so
> >> > > > > > > > lib/
> >> > > > > > > > lib_my_extension.so
> >> > > > > > > > include/
> >> > > > > > > > ... some headers maybe
> >> > > > > > > > xsd/
> >> > > > > > > > my_binding.xsd
> >> > > > > > > > any_other_folder/
> >> > > > > > > > any_stuff_I_like.xsd
> >> > > > > > > >
> >> > > > > > > > So the runtime will load any xsd that is in a folder
> named
> >> xsd
> >> > > and
> >> > > > > > > attempt
> >> > > > > > > > to load any library in a folder called extension. It
> would
> >> not
> >> > > > attempt
> >> > > > > > > to
> >> > > > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Cheers,
> >> > > > > > > >
> >> > > > > > > > --
> >> > > > > > > > Pete
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > Pete
> >> > > > > > >
> >> > > > > > > Sounds like the right idea to me. What goes in the bin
> dir?.
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > Anything you like!
> >> > > >
> >> > > >
> >> > > > On windows, .exe's and .dll's are often found in bin, whereas the
> >> > > > .lib's (used for linking against at compile time) are generally
> >> found
> >> > > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib
> >> (which
> >> > > > is more unix-like, and, IMO, nicer)
> >> > > >
> >> > > > >
> >> > > > > Should the
> >> > > > > > lib be...
> >> > > > > >
> >> > > > > > lib/
> >> > > > > > lib_my_extension.lib
> >> > > > >
> >> > > > >
> >> > > > > lif you like... again... anything you like.
> >> > > > >
> >> > > > > Maybe you could stick with bin,include, lib,xsd and then have
> >> your
> >> > > > "other"
> >> > > > > > directory have everything else in it that the extension might
> >> rely
> >> > > on.
> >> > > > > > Depends what you now anticipate being in bin and lib over and
> >> > above
> >> > > > the
> >> > > > > > actual library exposing the extension.
> >> > > > > >
> >> > > > > > Will you continue to mandate a specific place where
> extensions
> >> are
> >> > > > > > installed, i.e. from you post <sca_install_dir>/extensions
> >> folder?
> >> > > > >
> >> > > > >
> >> > > > > Yes
> >> > > > >
> >> > > > > So that
> >> > > > > > you can move an extension out of there if you don't want it
> to
> >> > load.
> >> > > > >
> >> > > > >
> >> > > > > Well the idea here is you don't need to remove the whole
> >> > package..just
> >> > > > the
> >> > > > > library from the extension dir
> >> > > > >
> >> > > > > Simon
> >> > > > > >
> >> > > > > >
> >> > > > > The layout of the extension package is whatever the package
> >> creater
> >> > > > wants to
> >> > > > > make it. We don't care. All we will care about is loading any
> >> > library
> >> > > in
> >> > > > a
> >> > > > > folder anywhere in the extension package that is in a folder
> >> called
> >> > > > > 'extension' and expecting it to implement the extension
> >> initialize
> >> > > > method.
> >> > > > > We will also load any schema files in any folder called
> >> 'xsd'. An
> >> > > > extension
> >> > > > > only needs the xsd folder if it has schema required for the
> >> runtime
> >> > > > model.
> >> > > > >
> >> > > > > As an example here is the ws binding extension implemented for
> >> > axis2c:
> >> > > > >
> >> > > > > <sca_install>/extensions/
> >> > > > > ... somewhere in this tree... could be anywhere...
> >> > > > > ws/
> >> > > > > xsd/
> >> > > > > ws-binding.xsd
> >> > > > > reference/
> >> > > > > lib/
> >> > > > > libtuscany_sca_reference.so
> >> > > > > service/
> >> > > > > lib/
> >> > > > > libtuscany_sca_service.so
> >> > > > >
> >> > > > >
> >> > > > > OK... today the runtime would load the ws-binding.xsd as it
> >> loads
> >> > any
> >> > > > xsd it
> >> > > > > finds. It would also load any .so it finds so both reference
> >> aand
> >> > > > service
> >> > > > > extensions would be loaded/initialized.
> >> > > > >
> >> > > > > With my proposed change the xsd would be loaded (as it's in a
> >> folder
> >> > > > named
> >> > > > > 'xsd' ) but the libraries would not. To get the extensions
> >> loaded
> >> > you
> >> > > > would
> >> > > > > have to package with an 'extension' folder ... again anywhere
> in
> >> the
> >> > > > tree...
> >> > > > > which "contains" the libraries to load eg.
> >> > > > >
> >> > > > > ws/
> >> > > > > extension/
> >> > > > > libload_this.so ->
> >> reference/lib/libtuscany_sca_reference.so
> >> > > > > xsd/
> >> > > > > ws-binding.xsd
> >> > > > > reference/
> >> > > > > lib/
> >> > > > > libtuscany_sca_reference.so
> >> > > > > service/
> >> > > > > lib/
> >> > > > > libtuscany_sca_service.so
> >> > > > >
> >> > > > > Here the runtime loads anything in any folder named
> >> 'extension' so
> >> > the
> >> > > > > reference extension would be loaded/initialized.
> >> > > >
> >> > > > Windows doesn't really do symlinks - I guess on windows we'd have
> >> the
> >> > > > tuscany_sca_reference.dll in extension/ and nothing in
> >> reference/lib/
> >> > > > or reference/bin/? For extensions that have dependencies on other
> >> > > > libraries, those dll's could go in lib/
> >> > >
> >> > >
> >> > > I would actually have a copy of the dll in extension on windows and
> >> > delete
> >> > > it to disable. For a large extension it would be very wasy to make
> a
> >> > tiny
> >> > > dll which just implements the initialise interface expected by the
> >> > runtime
> >> > > and leave everything else in a separate dll in bin/lib.
> >> > >
> >> > > > I would package the cpp language extension as
> >> > > > >
> >> > > > > cpp/
> >> > > > > bin/
> >> > > > > lib/
> >> > > > > include/
> >> > > > > xsd/
> >> > > > > extension/
> >> > > > >
> >> > > > > The bin, lib, include are exactly what you would expect from a
> >> > package
> >> > > > that
> >> > > > > you might want to build against. The extension/ folder would
> >> hold
> >> > the
> >> > > > > library that implements the extension interface and xsd
> holds...
> >> > well
> >> > > > xsds!
> >> > > > >
> >> > > >
> >> > > > All sounds good to me. +1
> >> > > >
> >> > > > Andy
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> >> > > > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> > > --
> >> > > Pete
> >> > >
> >> > > Ok, so I think I get it. My concern is that giving the developer
> >> the
> >> > flexibility to put "extension" and "xsd" anywhere they like means
> that
> >> > they
> >> > will accidentally duplicate these directories as these are pretty
> >> common
> >> > names for directories. So we could ask them always to have them at
> the
> >> top
> >> > level. Doesn't remove the possibility for duplication but reduces the
> >> risk
> >> > a
> >> > little. Or we could make the names a little more exotic with some
> >> > qualification "sca_extension", "sca_xsd" for example, and maintain
> the
> >> > flexibility you want.
> >> >
> >> > Simon
> >> >
> >> >
> >>
> >> Yes... the names are always the hardest thing to come up with! They
> >> should
> >> not be sca_ as this is Tuscany specific.
> >>
> >> so... we could get away with just the one folder tuscany_extension that
> >> contains the schema and libraries:
> >>
> >>
> >> cpp/
> >> bin/
> >> lib/
> >> include/
> >> xsd/
> >> tuscany_extension/ <- schema and libraries implementing extension
> i/f
> >> live under this tree
> >> xsd -> ../xsd <- could even be a symlink to where they really
> are
> >>
> >> I like that better. Runtime will load library and schema from any
> >> subtree
> >> under <sca_install>/extensions that is named 'tuscany_extension' and
> >> ignore
> >> anything else.
> >>
> >> Nice and flexible.
> >>
> >>
> >> --
> >> Pete
> >>
> >> That sounds like a plan. Nice one.
> >
> > Simon
> >
>
> Wow, 15 messages on this thread since last evening :) The latest
> proposal looks really good to me, nice, flexible and simple!
>
> I would only propose to shorten "tuscany_extension" to something like
> "module" as we are already under an "extensions" directory maybe? This
> terminology would be similar to Apache httpd and Axis2C as well, but
> this is really nitpicking so if you prefer "tuscany_extension" I'm
> perfectly fine with it too. I could even convince myself that I actually
> prefer "tuscany_extension" :) I just wanted to mention it to see what
> you guys think...
as I said, the naming is the hardest part!!!
tuscany_extension is too long. I would have gone with just extension but I
think module is the best so far.
Cheers,
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Simon Laws wrote:
> On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>>
>> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
>> >
>> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>> > >
>> > > On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
>> > > >
>> > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>> > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
>> > > > > >
>> > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>> > > > > > >
>> > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On 30/11/06, Simon Laws <si...@googlemail.com>
>> wrote:
>> > > > > > > > >
>> > > > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com>
>> wrote:
>> > > > > > > > > >
>> > > > > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com>
>> > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
>> > > jsdelfino@apache.org>
>> > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > Pete Robbins wrote:
>> > > > > > > > > > > > > Our current method of packaging and loading an
>> > > extension
>> > > > is
>> > > > > > > > > fairly
>> > > > > > > > > > > > > simple:
>> > > > > > > > > > > > > we load all schema and libraries in the
>> extensions
>> > > path.
>> > > > > > This
>> > > > > > > > > has a
>> > > > > > > > > > > > > number
>> > > > > > > > > > > > > of problems.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 1. An extension may consist of more than one
>> library
>> > > e.g
>> > > > .
>> > > > > > > > > > > > > libmy_extension.so
>> > > > > > > > > > > > > and libmy_extension_utils.so. Our current
>> loading
>> > > scheme
>> > > > > > will
>> > > > > > > > > > attempt
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > load both of these and may fail on the one that
>> > > doesn't
>> > > > > > > provide
>> > > > > > > > > the
>> > > > > > > > > > > > > extension initialize method. On MacOS the output
>> of
>> > > our
>> > > > > > build
>> > > > > > > > > > produces
>> > > > > > > > > > > > a
>> > > > > > > > > > > > > libx.dylib and a series of symlinks to this
>> called
>> > > > > > > libx.0.dylib,
>> > > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of
>> these
>> > > which
>> > > > > > > doesn't
>> > > > > > > > > > cause
>> > > > > > > > > > > > > problems as they are all the sam library and
>> just
>> > > > repeatedly
>> > > > > > > > > > register
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > handle the same requests. Very inefficient
>> though!
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 2. Control of whether or not to load an
>> extension
>> > > > library is
>> > > > > > > > > > currently
>> > > > > > > > > > > > by
>> > > > > > > > > > > > > renaming the library so the runtime doesn't find
>> it.
>> > > An
>> > > > > > > example
>> > > > > > > > > is
>> > > > > > > > > > > > > that we
>> > > > > > > > > > > > > ship our python extension as
>> > > > > > libtuscany_sca_python.so.diabled.
>> > > > > > > > > This
>> > > > > > > > > > is
>> > > > > > > > > > > >
>> > > > > > > > > > > > > horrible and error prone.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > We could improve this by having a system
>> > configuration
>> > > > file
>> > > > > > > that
>> > > > > > > > > > lists
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > required extensions but the I like the self
>> > contained
>> > > > > > package
>> > > > > > > > > > approach
>> > > > > > > > > > > > > that
>> > > > > > > > > > > > > we have now. I'd like to implement an improved
>> > scheme
>> > > > for
>> > > > > > > > > packaging
>> > > > > > > > > > an
>> > > > > > > > > > > > > extension by introducing a per extension
>> > configuration
>> > > > file.
>> > > > > > > > > > Something
>> > > > > > > > > > > > > like:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > tuscany_sca_ws_binding.extension
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > <scacpp:extension name="ws binding"
>> enabled="true">
>> > > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
>> > > > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
>> > > > > > > > > > > > > </scacppp:extension>
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > So the package would look like:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > extensions/
>> > > > > > > > > > > > > ws/
>> > > > > > > > > > > > > tusany_sca_ws_binding.extension
>> > > > > > > > > > > > > lib/
>> > > > > > > > > > > > > xsd/
>> > > > > > > > > > > > > other_folder/
>> > > > > > > > > > > > > ...
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The .extension configuration file is saying to
>> load
>> > > the
>> > > > > > > library
>> > > > > > > > > > which
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > located somewhere in the package... the runtime
>> will
>> > > > find
>> > > > > > > it...
>> > > > > > > > > no
>> > > > > > > > > > > > > need to
>> > > > > > > > > > > > > specify a path.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Taking this further the configuration file could
>> > list
>> > > > the
>> > > > > > > schema
>> > > > > > > > > to
>> > > > > > > > > > be
>> > > > > > > > > > > > > loaded. Currently the runtime will just load any
>> it
>> > > > finds
>> > > > > > but
>> > > > > > > > > these
>> > > > > > > > > > > > > may not
>> > > > > > > > > > > > > be needed by the runtime e.g. the schema may be
>> for
>> > > some
>> > > > > > > > > extension
>> > > > > > > > > > > > > implementation specific purpose.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > I think it would also be good for the extension
>> > > > > > > initialization()
>> > > > > > > > > > > > > method to
>> > > > > > > > > > > > > take as a parameter the root of the extension
>> e.g.
>> > > > > > > > > > > > > extension("/tuscany/extensions/ws"). This would
>> > allow
>> > > > the
>> > > > > > > > > extension
>> > > > > > > > > > > > > package
>> > > > > > > > > > > > > to contain any configuration information that it
>> > > needs.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > I'd like to start by at least introducing the
>> > > .extension
>> > > > > > file
>> > > > > > > > > for
>> > > > > > > > > > each
>> > > > > > > > > > > > > extension and loading only the specified
>> > library(ies)
>> > > if
>> > > > the
>> > > > > > > > > > extension
>> > > > > > > > > > > > is
>> > > > > > > > > > > > > enabled.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Any thoughts?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > Two thoughts:
>> > > > > > > > > > > > - convention over configuration
>> > > > > > > > > > > > - the runtime should be consumable without
>> having to
>> > go
>> > > > tweak
>> > > > > > > XML
>> > > > > > > > > > > > configuration files
>> > > > > > > > > > > >
>> > > > > > > > > > > > If I remember correctly, renaming the dlls to
>> > .disabled
>> > > > was a
>> > > > > > > last
>> > > > > > > > >
>> > > > > > > > > > > > minute change to work around DLL loading errors
>> with
>> > our
>> > > > M2
>> > > > > > > > > release on
>> > > > > > > > > > > > Windows.
>> > > > > > > > > > > >
>> > > > > > > > > > > > I agree that we should do better than renaming to
>> > > > .disabled,
>> > > > > > but
>> > > > > > > > > I'd
>> > > > > > > > > > > > like to understand better the actual issues
>> that we
>> > > faced
>> > > > > > before
>> > > > > > > > > > > > inventing yet another XML based
>> runtime configuration
>> > > > > > language
>> > > > > > > :)
>> > > > > > > > > > > >
>> > > > > > > > > > > > I'm aware of the following issues:
>> > > > > > > > > > > > 1. We need the runtime to load extension libs
>> only,
>> > not
>> > > > other
>> > > > > > > libs
>> > > > > > > > > > under
>> > > > > > > > > > > >
>> > > > > > > > > > > > the extension directory which are not actual
>> > extensions
>> > > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that
>> contribute
>> > > to
>> > > > > > SCDL,
>> > > > > > > > > and
>> > > > > > > > > > > > leave other XSDs under the extension directory
>> alone
>> > > > > > > > > > > > 3. Extensions that cannot be loaded because
>> some of
>> > > their
>> > > > > > > > > dependencies
>> > > > > > > > > > > > are not there should no break the runtime
>> > > > > > > > > > > > 4. The system admin / installer should be able to
>> > > disable
>> > > > > > > > > extensions
>> > > > > > > > > > > > that won't load because their dependencies are not
>> > there
>> > > > (I'm
>> > > > > > > not
>> > > > > > > > > yet
>> > > > > > > > > > > > convinced that this is still an issue if we manage
>> to
>> > > > solve
>> > > > > > > issue
>> > > > > > > > > #3)
>> > > > > > > > > > > >
>> > > > > > > > > > > > Did we run into any other big issues?
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > No, that's about it.
>> > > > > > > > > > >
>> > > > > > > > > > > 2. XSD loading can be done by convention (schemas
>> for
>> > the
>> > > > > > runtime
>> > > > > > > > > are
>> > > > > > > > > > > always in a folder called 'xsd'
>> > > > > > > > > > > 1. Could be solved in a similar way by only loading
>> > > > libraries in
>> > > > > > a
>> > > > > > > > > > folder
>> > > > > > > > > > > called ???
>> > > > > > > > > > > 3. Can be solved by just ignoring the load
>> > errors/issuing
>> > > a
>> > > > > > > warning
>> > > > > > > > > > rather
>> > > > > > > > > > > than giving up
>> > > > > > > > > > > 4. Can be solved by solution to 3.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > I recall now why 3. was a big problem. Windows
>> sometimes
>> > > > throws up
>> > > > > > > an
>> > > > > > > > > > error
>> > > > > > > > > > dialog when the load fails so it was not just a
>> case of
>> > the
>> > > > > > runtime
>> > > > > > > > > > handling
>> > > > > > > > > > the load failure so we had to "disable" the extension.
>> > > > > > > > > >
>> > > > > > > > > > Cheers,
>> > > > > > > > > >
>> > > > > > > > > > --
>> > > > > > > > > > Pete
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > --
>> > > > > > > > > > Pete
>> > > > > > > > > >
>> > > > > > > > > > Pete, just summarizing so that I understand. It seems
>> > there
>> > > > are
>> > > > > > two
>> > > > > > > > > parts
>> > > > > > > > > to this.
>> > > > > > > > >
>> > > > > > > > > A/ Building and installing an extension in order that it
>> can
>> > > be
>> > > > > > > consumed
>> > > > > > > > > by
>> > > > > > > > > the runtime. Building and installing extensions is
>> > potentially
>> > > a
>> > > > > > > > > separate
>> > > > > > > > > activity from building and installing the runtime
>> itself.
>> > > > Looking at
>> > > > > > > the
>> > > > > > > > > M2
>> > > > > > > > > download they all come together at the moment.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Yes. It just happens that we are the only providers of
>> > > extensions
>> > > > at
>> > > > > > the
>> > > > > > > > moment.
>> > > > > > > >
>> > > > > > > > B/ Optionally consuming/enabling an extension in the
>> runtime,
>> > > once
>> > > > the
>> > > > > > > > > extension has been installed, in such a way that the
>> runtime
>> > > is
>> > > > able
>> > > > > > > to
>> > > > > > > > > find
>> > > > > > > > > and load it.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Yes.
>> > > > > > > >
>> > > > > > > > If I understand the previous posts (and the current
>> code) A/
>> > is
>> > > > > > achieved
>> > > > > > > > > by
>> > > > > > > > > building the extension in the deployed folder layout
>> > > > > > > > > extension type/
>> > > > > > > > > bin
>> > > > > > > > > lib
>> > > > > > > > > xsd
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > The current runtime does not mandate any layout. It will
>> seek
>> > > and
>> > > > > > > > destroy.. I mean seek and load any library it finds and
>> any
>> > .xsd
>> > > > it
>> > > > > > > finds.
>> > > > > > > > That layout is just how we package extensions in ou
>> build at
>> > the
>> > > > > > moment.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > and Pete you are proposing that this is adjusted so that
>> there
>> > > are
>> > > > > > > > > directories that hold just the stuff that the
>> contains the
>> > > > extension
>> > > > > > > > > rather
>> > > > > > > > > than the things it depends on.
>> > > > > > > > > extension type/
>> > > > > > > > > bin
>> > > > > > > > > extensionbin?
>> > > > > > > > > lib
>> > > > > > > > > xsd
>> > > > > > > > > extensionxsd?
>> > > > > > > > >
>> > > > > > > > > You are also proposing that B/ is achieved by ensuring
>> that
>> > > this
>> > > > > > > > > directory
>> > > > > > > > > structure be placed in a location that the the
>> runtime can
>> > > > search
>> > > > > > for
>> > > > > > > > > active
>> > > > > > > > > extensions (as it does at the moment in the deploy
>> > directory)
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > it currently loads any extensions from under the
>> > > > > > > > <sca_install_dir>/extensions folder
>> > > > > > > >
>> > > > > > > > and that the
>> > > > > > > > > runtime ignores any badly configured extensions if
>> possible.
>> > > > > > Removing
>> > > > > > > an
>> > > > > > > > > extension's directory from the deploy
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > extensions not deploy
>> > > > > > > >
>> > > > > > > > directory structure has the effect of
>> > > > > > > > > disabling it so I guess this is the fallback if the
>> runtime
>> > > > can't
>> > > > > > > > > continue.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > That is true in today's implementation.
>> > > > > > > >
>> > > > > > > > Anything more complex and, as Jean-Sebastien suggests, you
>> > start
>> > > > > > getting
>> > > > > > > > > into the full blown package and dependency management
>> > problem
>> > > > that
>> > > > > > > many
>> > > > > > > > > other systems try to solve in different ways, e.g.
>> rpm or
>> > > pear.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Agreed... I'll give up on that plan ;-)
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > What we need is to enable the runtime to identify which
>> > > extension
>> > > > > > > > libraries to load and which schema to load. My suggestion
>> is:
>> > > > > > > >
>> > > > > > > > my_extension/
>> > > > > > > > bin/
>> > > > > > > > extension/
>> > > > > > > > library_that_will_be_loaded_by_the_runtime.so
>> > > > > > > > lib/
>> > > > > > > > lib_my_extension.so
>> > > > > > > > include/
>> > > > > > > > ... some headers maybe
>> > > > > > > > xsd/
>> > > > > > > > my_binding.xsd
>> > > > > > > > any_other_folder/
>> > > > > > > > any_stuff_I_like.xsd
>> > > > > > > >
>> > > > > > > > So the runtime will load any xsd that is in a folder named
>> xsd
>> > > and
>> > > > > > > attempt
>> > > > > > > > to load any library in a folder called extension. It would
>> not
>> > > > attempt
>> > > > > > > to
>> > > > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Cheers,
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > Pete
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Pete
>> > > > > > >
>> > > > > > > Sounds like the right idea to me. What goes in the bin dir?.
>> > > > >
>> > > > >
>> > > > >
>> > > > > Anything you like!
>> > > >
>> > > >
>> > > > On windows, .exe's and .dll's are often found in bin, whereas the
>> > > > .lib's (used for linking against at compile time) are generally
>> found
>> > > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib
>> (which
>> > > > is more unix-like, and, IMO, nicer)
>> > > >
>> > > > >
>> > > > > Should the
>> > > > > > lib be...
>> > > > > >
>> > > > > > lib/
>> > > > > > lib_my_extension.lib
>> > > > >
>> > > > >
>> > > > > lif you like... again... anything you like.
>> > > > >
>> > > > > Maybe you could stick with bin,include, lib,xsd and then have
>> your
>> > > > "other"
>> > > > > > directory have everything else in it that the extension might
>> rely
>> > > on.
>> > > > > > Depends what you now anticipate being in bin and lib over and
>> > above
>> > > > the
>> > > > > > actual library exposing the extension.
>> > > > > >
>> > > > > > Will you continue to mandate a specific place where extensions
>> are
>> > > > > > installed, i.e. from you post <sca_install_dir>/extensions
>> folder?
>> > > > >
>> > > > >
>> > > > > Yes
>> > > > >
>> > > > > So that
>> > > > > > you can move an extension out of there if you don't want it to
>> > load.
>> > > > >
>> > > > >
>> > > > > Well the idea here is you don't need to remove the whole
>> > package..just
>> > > > the
>> > > > > library from the extension dir
>> > > > >
>> > > > > Simon
>> > > > > >
>> > > > > >
>> > > > > The layout of the extension package is whatever the package
>> creater
>> > > > wants to
>> > > > > make it. We don't care. All we will care about is loading any
>> > library
>> > > in
>> > > > a
>> > > > > folder anywhere in the extension package that is in a folder
>> called
>> > > > > 'extension' and expecting it to implement the extension
>> initialize
>> > > > method.
>> > > > > We will also load any schema files in any folder called
>> 'xsd'. An
>> > > > extension
>> > > > > only needs the xsd folder if it has schema required for the
>> runtime
>> > > > model.
>> > > > >
>> > > > > As an example here is the ws binding extension implemented for
>> > axis2c:
>> > > > >
>> > > > > <sca_install>/extensions/
>> > > > > ... somewhere in this tree... could be anywhere...
>> > > > > ws/
>> > > > > xsd/
>> > > > > ws-binding.xsd
>> > > > > reference/
>> > > > > lib/
>> > > > > libtuscany_sca_reference.so
>> > > > > service/
>> > > > > lib/
>> > > > > libtuscany_sca_service.so
>> > > > >
>> > > > >
>> > > > > OK... today the runtime would load the ws-binding.xsd as it
>> loads
>> > any
>> > > > xsd it
>> > > > > finds. It would also load any .so it finds so both reference
>> aand
>> > > > service
>> > > > > extensions would be loaded/initialized.
>> > > > >
>> > > > > With my proposed change the xsd would be loaded (as it's in a
>> folder
>> > > > named
>> > > > > 'xsd' ) but the libraries would not. To get the extensions
>> loaded
>> > you
>> > > > would
>> > > > > have to package with an 'extension' folder ... again anywhere in
>> the
>> > > > tree...
>> > > > > which "contains" the libraries to load eg.
>> > > > >
>> > > > > ws/
>> > > > > extension/
>> > > > > libload_this.so ->
>> reference/lib/libtuscany_sca_reference.so
>> > > > > xsd/
>> > > > > ws-binding.xsd
>> > > > > reference/
>> > > > > lib/
>> > > > > libtuscany_sca_reference.so
>> > > > > service/
>> > > > > lib/
>> > > > > libtuscany_sca_service.so
>> > > > >
>> > > > > Here the runtime loads anything in any folder named
>> 'extension' so
>> > the
>> > > > > reference extension would be loaded/initialized.
>> > > >
>> > > > Windows doesn't really do symlinks - I guess on windows we'd have
>> the
>> > > > tuscany_sca_reference.dll in extension/ and nothing in
>> reference/lib/
>> > > > or reference/bin/? For extensions that have dependencies on other
>> > > > libraries, those dll's could go in lib/
>> > >
>> > >
>> > > I would actually have a copy of the dll in extension on windows and
>> > delete
>> > > it to disable. For a large extension it would be very wasy to make a
>> > tiny
>> > > dll which just implements the initialise interface expected by the
>> > runtime
>> > > and leave everything else in a separate dll in bin/lib.
>> > >
>> > > > I would package the cpp language extension as
>> > > > >
>> > > > > cpp/
>> > > > > bin/
>> > > > > lib/
>> > > > > include/
>> > > > > xsd/
>> > > > > extension/
>> > > > >
>> > > > > The bin, lib, include are exactly what you would expect from a
>> > package
>> > > > that
>> > > > > you might want to build against. The extension/ folder would
>> hold
>> > the
>> > > > > library that implements the extension interface and xsd holds...
>> > well
>> > > > xsds!
>> > > > >
>> > > >
>> > > > All sounds good to me. +1
>> > > >
>> > > > Andy
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> > > > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>> > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Pete
>> > >
>> > > Ok, so I think I get it. My concern is that giving the developer
>> the
>> > flexibility to put "extension" and "xsd" anywhere they like means that
>> > they
>> > will accidentally duplicate these directories as these are pretty
>> common
>> > names for directories. So we could ask them always to have them at the
>> top
>> > level. Doesn't remove the possibility for duplication but reduces the
>> risk
>> > a
>> > little. Or we could make the names a little more exotic with some
>> > qualification "sca_extension", "sca_xsd" for example, and maintain the
>> > flexibility you want.
>> >
>> > Simon
>> >
>> >
>>
>> Yes... the names are always the hardest thing to come up with! They
>> should
>> not be sca_ as this is Tuscany specific.
>>
>> so... we could get away with just the one folder tuscany_extension that
>> contains the schema and libraries:
>>
>>
>> cpp/
>> bin/
>> lib/
>> include/
>> xsd/
>> tuscany_extension/ <- schema and libraries implementing extension i/f
>> live under this tree
>> xsd -> ../xsd <- could even be a symlink to where they really are
>>
>> I like that better. Runtime will load library and schema from any
>> subtree
>> under <sca_install>/extensions that is named 'tuscany_extension' and
>> ignore
>> anything else.
>>
>> Nice and flexible.
>>
>>
>> --
>> Pete
>>
>> That sounds like a plan. Nice one.
>
> Simon
>
Wow, 15 messages on this thread since last evening :) The latest
proposal looks really good to me, nice, flexible and simple!
I would only propose to shorten "tuscany_extension" to something like
"module" as we are already under an "extensions" directory maybe? This
terminology would be similar to Apache httpd and Axis2C as well, but
this is really nitpicking so if you prefer "tuscany_extension" I'm
perfectly fine with it too. I could even convince myself that I actually
prefer "tuscany_extension" :) I just wanted to mention it to see what
you guys think...
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: [C++] Binding/Language Extensions - packaging
Posted by Simon Laws <si...@googlemail.com>.
On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>
> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> >
> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >
> > > On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
> > > >
> > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > >
> > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > >
> > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > > > > >
> > > > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com>
> wrote:
> > > > > > > > > >
> > > > > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com>
> > wrote:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
> > > jsdelfino@apache.org>
> > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Pete Robbins wrote:
> > > > > > > > > > > > > Our current method of packaging and loading an
> > > extension
> > > > is
> > > > > > > > > fairly
> > > > > > > > > > > > > simple:
> > > > > > > > > > > > > we load all schema and libraries in the extensions
> > > path.
> > > > > > This
> > > > > > > > > has a
> > > > > > > > > > > > > number
> > > > > > > > > > > > > of problems.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 1. An extension may consist of more than one
> library
> > > e.g
> > > > .
> > > > > > > > > > > > > libmy_extension.so
> > > > > > > > > > > > > and libmy_extension_utils.so. Our current loading
> > > scheme
> > > > > > will
> > > > > > > > > > attempt
> > > > > > > > > > > > to
> > > > > > > > > > > > > load both of these and may fail on the one that
> > > doesn't
> > > > > > > provide
> > > > > > > > > the
> > > > > > > > > > > > > extension initialize method. On MacOS the output
> of
> > > our
> > > > > > build
> > > > > > > > > > produces
> > > > > > > > > > > > a
> > > > > > > > > > > > > libx.dylib and a series of symlinks to this called
> > > > > > > libx.0.dylib,
> > > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these
> > > which
> > > > > > > doesn't
> > > > > > > > > > cause
> > > > > > > > > > > > > problems as they are all the sam library and just
> > > > repeatedly
> > > > > > > > > > register
> > > > > > > > > > > > to
> > > > > > > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2. Control of whether or not to load an extension
> > > > library is
> > > > > > > > > > currently
> > > > > > > > > > > > by
> > > > > > > > > > > > > renaming the library so the runtime doesn't find
> it.
> > > An
> > > > > > > example
> > > > > > > > > is
> > > > > > > > > > > > > that we
> > > > > > > > > > > > > ship our python extension as
> > > > > > libtuscany_sca_python.so.diabled.
> > > > > > > > > This
> > > > > > > > > > is
> > > > > > > > > > > >
> > > > > > > > > > > > > horrible and error prone.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We could improve this by having a system
> > configuration
> > > > file
> > > > > > > that
> > > > > > > > > > lists
> > > > > > > > > > > > > the
> > > > > > > > > > > > > required extensions but the I like the self
> > contained
> > > > > > package
> > > > > > > > > > approach
> > > > > > > > > > > > > that
> > > > > > > > > > > > > we have now. I'd like to implement an improved
> > scheme
> > > > for
> > > > > > > > > packaging
> > > > > > > > > > an
> > > > > > > > > > > > > extension by introducing a per extension
> > configuration
> > > > file.
> > > > > > > > > > Something
> > > > > > > > > > > > > like:
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > <scacpp:extension name="ws binding"
> enabled="true">
> > > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > > > > > > </scacppp:extension>
> > > > > > > > > > > > >
> > > > > > > > > > > > > So the package would look like:
> > > > > > > > > > > > >
> > > > > > > > > > > > > extensions/
> > > > > > > > > > > > > ws/
> > > > > > > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > > > > > > lib/
> > > > > > > > > > > > > xsd/
> > > > > > > > > > > > > other_folder/
> > > > > > > > > > > > > ...
> > > > > > > > > > > > >
> > > > > > > > > > > > > The .extension configuration file is saying to
> load
> > > the
> > > > > > > library
> > > > > > > > > > which
> > > > > > > > > > > > is
> > > > > > > > > > > > > located somewhere in the package... the runtime
> will
> > > > find
> > > > > > > it...
> > > > > > > > > no
> > > > > > > > > > > > > need to
> > > > > > > > > > > > > specify a path.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Taking this further the configuration file could
> > list
> > > > the
> > > > > > > schema
> > > > > > > > > to
> > > > > > > > > > be
> > > > > > > > > > > > > loaded. Currently the runtime will just load any
> it
> > > > finds
> > > > > > but
> > > > > > > > > these
> > > > > > > > > > > > > may not
> > > > > > > > > > > > > be needed by the runtime e.g. the schema may be
> for
> > > some
> > > > > > > > > extension
> > > > > > > > > > > > > implementation specific purpose.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think it would also be good for the extension
> > > > > > > initialization()
> > > > > > > > > > > > > method to
> > > > > > > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > > > > > > extension("/tuscany/extensions/ws"). This would
> > allow
> > > > the
> > > > > > > > > extension
> > > > > > > > > > > > > package
> > > > > > > > > > > > > to contain any configuration information that it
> > > needs.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd like to start by at least introducing the
> > > .extension
> > > > > > file
> > > > > > > > > for
> > > > > > > > > > each
> > > > > > > > > > > > > extension and loading only the specified
> > library(ies)
> > > if
> > > > the
> > > > > > > > > > extension
> > > > > > > > > > > > is
> > > > > > > > > > > > > enabled.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Two thoughts:
> > > > > > > > > > > > - convention over configuration
> > > > > > > > > > > > - the runtime should be consumable without having to
> > go
> > > > tweak
> > > > > > > XML
> > > > > > > > > > > > configuration files
> > > > > > > > > > > >
> > > > > > > > > > > > If I remember correctly, renaming the dlls to
> > .disabled
> > > > was a
> > > > > > > last
> > > > > > > > >
> > > > > > > > > > > > minute change to work around DLL loading errors with
> > our
> > > > M2
> > > > > > > > > release on
> > > > > > > > > > > > Windows.
> > > > > > > > > > > >
> > > > > > > > > > > > I agree that we should do better than renaming to
> > > > .disabled,
> > > > > > but
> > > > > > > > > I'd
> > > > > > > > > > > > like to understand better the actual issues that we
> > > faced
> > > > > > before
> > > > > > > > > > > > inventing yet another XML based
> runtime configuration
> > > > > > language
> > > > > > > :)
> > > > > > > > > > > >
> > > > > > > > > > > > I'm aware of the following issues:
> > > > > > > > > > > > 1. We need the runtime to load extension libs only,
> > not
> > > > other
> > > > > > > libs
> > > > > > > > > > under
> > > > > > > > > > > >
> > > > > > > > > > > > the extension directory which are not actual
> > extensions
> > > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that
> contribute
> > > to
> > > > > > SCDL,
> > > > > > > > > and
> > > > > > > > > > > > leave other XSDs under the extension directory alone
> > > > > > > > > > > > 3. Extensions that cannot be loaded because some of
> > > their
> > > > > > > > > dependencies
> > > > > > > > > > > > are not there should no break the runtime
> > > > > > > > > > > > 4. The system admin / installer should be able to
> > > disable
> > > > > > > > > extensions
> > > > > > > > > > > > that won't load because their dependencies are not
> > there
> > > > (I'm
> > > > > > > not
> > > > > > > > > yet
> > > > > > > > > > > > convinced that this is still an issue if we manage
> to
> > > > solve
> > > > > > > issue
> > > > > > > > > #3)
> > > > > > > > > > > >
> > > > > > > > > > > > Did we run into any other big issues?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > No, that's about it.
> > > > > > > > > > >
> > > > > > > > > > > 2. XSD loading can be done by convention (schemas for
> > the
> > > > > > runtime
> > > > > > > > > are
> > > > > > > > > > > always in a folder called 'xsd'
> > > > > > > > > > > 1. Could be solved in a similar way by only loading
> > > > libraries in
> > > > > > a
> > > > > > > > > > folder
> > > > > > > > > > > called ???
> > > > > > > > > > > 3. Can be solved by just ignoring the load
> > errors/issuing
> > > a
> > > > > > > warning
> > > > > > > > > > rather
> > > > > > > > > > > than giving up
> > > > > > > > > > > 4. Can be solved by solution to 3.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I recall now why 3. was a big problem. Windows sometimes
> > > > throws up
> > > > > > > an
> > > > > > > > > > error
> > > > > > > > > > dialog when the load fails so it was not just a case of
> > the
> > > > > > runtime
> > > > > > > > > > handling
> > > > > > > > > > the load failure so we had to "disable" the extension.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Pete
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Pete
> > > > > > > > > >
> > > > > > > > > > Pete, just summarizing so that I understand. It seems
> > there
> > > > are
> > > > > > two
> > > > > > > > > parts
> > > > > > > > > to this.
> > > > > > > > >
> > > > > > > > > A/ Building and installing an extension in order that it
> can
> > > be
> > > > > > > consumed
> > > > > > > > > by
> > > > > > > > > the runtime. Building and installing extensions is
> > potentially
> > > a
> > > > > > > > > separate
> > > > > > > > > activity from building and installing the runtime itself.
> > > > Looking at
> > > > > > > the
> > > > > > > > > M2
> > > > > > > > > download they all come together at the moment.
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes. It just happens that we are the only providers of
> > > extensions
> > > > at
> > > > > > the
> > > > > > > > moment.
> > > > > > > >
> > > > > > > > B/ Optionally consuming/enabling an extension in the
> runtime,
> > > once
> > > > the
> > > > > > > > > extension has been installed, in such a way that the
> runtime
> > > is
> > > > able
> > > > > > > to
> > > > > > > > > find
> > > > > > > > > and load it.
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes.
> > > > > > > >
> > > > > > > > If I understand the previous posts (and the current code) A/
> > is
> > > > > > achieved
> > > > > > > > > by
> > > > > > > > > building the extension in the deployed folder layout
> > > > > > > > > extension type/
> > > > > > > > > bin
> > > > > > > > > lib
> > > > > > > > > xsd
> > > > > > > >
> > > > > > > >
> > > > > > > > The current runtime does not mandate any layout. It will
> seek
> > > and
> > > > > > > > destroy.. I mean seek and load any library it finds and any
> > .xsd
> > > > it
> > > > > > > finds.
> > > > > > > > That layout is just how we package extensions in ou build at
> > the
> > > > > > moment.
> > > > > > > >
> > > > > > > >
> > > > > > > > and Pete you are proposing that this is adjusted so that
> there
> > > are
> > > > > > > > > directories that hold just the stuff that the contains the
> > > > extension
> > > > > > > > > rather
> > > > > > > > > than the things it depends on.
> > > > > > > > > extension type/
> > > > > > > > > bin
> > > > > > > > > extensionbin?
> > > > > > > > > lib
> > > > > > > > > xsd
> > > > > > > > > extensionxsd?
> > > > > > > > >
> > > > > > > > > You are also proposing that B/ is achieved by ensuring
> that
> > > this
> > > > > > > > > directory
> > > > > > > > > structure be placed in a location that the the runtime can
> > > > search
> > > > > > for
> > > > > > > > > active
> > > > > > > > > extensions (as it does at the moment in the deploy
> > directory)
> > > > > > > >
> > > > > > > >
> > > > > > > > it currently loads any extensions from under the
> > > > > > > > <sca_install_dir>/extensions folder
> > > > > > > >
> > > > > > > > and that the
> > > > > > > > > runtime ignores any badly configured extensions if
> possible.
> > > > > > Removing
> > > > > > > an
> > > > > > > > > extension's directory from the deploy
> > > > > > > >
> > > > > > > >
> > > > > > > > extensions not deploy
> > > > > > > >
> > > > > > > > directory structure has the effect of
> > > > > > > > > disabling it so I guess this is the fallback if the
> runtime
> > > > can't
> > > > > > > > > continue.
> > > > > > > >
> > > > > > > >
> > > > > > > > That is true in today's implementation.
> > > > > > > >
> > > > > > > > Anything more complex and, as Jean-Sebastien suggests, you
> > start
> > > > > > getting
> > > > > > > > > into the full blown package and dependency management
> > problem
> > > > that
> > > > > > > many
> > > > > > > > > other systems try to solve in different ways, e.g. rpm or
> > > pear.
> > > > > > > >
> > > > > > > >
> > > > > > > > Agreed... I'll give up on that plan ;-)
> > > > > > > >
> > > > > > > >
> > > > > > > > What we need is to enable the runtime to identify which
> > > extension
> > > > > > > > libraries to load and which schema to load. My suggestion
> is:
> > > > > > > >
> > > > > > > > my_extension/
> > > > > > > > bin/
> > > > > > > > extension/
> > > > > > > > library_that_will_be_loaded_by_the_runtime.so
> > > > > > > > lib/
> > > > > > > > lib_my_extension.so
> > > > > > > > include/
> > > > > > > > ... some headers maybe
> > > > > > > > xsd/
> > > > > > > > my_binding.xsd
> > > > > > > > any_other_folder/
> > > > > > > > any_stuff_I_like.xsd
> > > > > > > >
> > > > > > > > So the runtime will load any xsd that is in a folder named
> xsd
> > > and
> > > > > > > attempt
> > > > > > > > to load any library in a folder called extension. It would
> not
> > > > attempt
> > > > > > > to
> > > > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > > > > > > >
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > --
> > > > > > > > Pete
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Pete
> > > > > > >
> > > > > > > Sounds like the right idea to me. What goes in the bin dir?.
> > > > >
> > > > >
> > > > >
> > > > > Anything you like!
> > > >
> > > >
> > > > On windows, .exe's and .dll's are often found in bin, whereas the
> > > > .lib's (used for linking against at compile time) are generally
> found
> > > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib
> (which
> > > > is more unix-like, and, IMO, nicer)
> > > >
> > > > >
> > > > > Should the
> > > > > > lib be...
> > > > > >
> > > > > > lib/
> > > > > > lib_my_extension.lib
> > > > >
> > > > >
> > > > > lif you like... again... anything you like.
> > > > >
> > > > > Maybe you could stick with bin,include, lib,xsd and then have your
> > > > "other"
> > > > > > directory have everything else in it that the extension might
> rely
> > > on.
> > > > > > Depends what you now anticipate being in bin and lib over and
> > above
> > > > the
> > > > > > actual library exposing the extension.
> > > > > >
> > > > > > Will you continue to mandate a specific place where extensions
> are
> > > > > > installed, i.e. from you post <sca_install_dir>/extensions
> folder?
> > > > >
> > > > >
> > > > > Yes
> > > > >
> > > > > So that
> > > > > > you can move an extension out of there if you don't want it to
> > load.
> > > > >
> > > > >
> > > > > Well the idea here is you don't need to remove the whole
> > package..just
> > > > the
> > > > > library from the extension dir
> > > > >
> > > > > Simon
> > > > > >
> > > > > >
> > > > > The layout of the extension package is whatever the package
> creater
> > > > wants to
> > > > > make it. We don't care. All we will care about is loading any
> > library
> > > in
> > > > a
> > > > > folder anywhere in the extension package that is in a folder
> called
> > > > > 'extension' and expecting it to implement the extension initialize
> > > > method.
> > > > > We will also load any schema files in any folder called 'xsd'. An
> > > > extension
> > > > > only needs the xsd folder if it has schema required for the
> runtime
> > > > model.
> > > > >
> > > > > As an example here is the ws binding extension implemented for
> > axis2c:
> > > > >
> > > > > <sca_install>/extensions/
> > > > > ... somewhere in this tree... could be anywhere...
> > > > > ws/
> > > > > xsd/
> > > > > ws-binding.xsd
> > > > > reference/
> > > > > lib/
> > > > > libtuscany_sca_reference.so
> > > > > service/
> > > > > lib/
> > > > > libtuscany_sca_service.so
> > > > >
> > > > >
> > > > > OK... today the runtime would load the ws-binding.xsd as it loads
> > any
> > > > xsd it
> > > > > finds. It would also load any .so it finds so both reference aand
> > > > service
> > > > > extensions would be loaded/initialized.
> > > > >
> > > > > With my proposed change the xsd would be loaded (as it's in a
> folder
> > > > named
> > > > > 'xsd' ) but the libraries would not. To get the extensions loaded
> > you
> > > > would
> > > > > have to package with an 'extension' folder ... again anywhere in
> the
> > > > tree...
> > > > > which "contains" the libraries to load eg.
> > > > >
> > > > > ws/
> > > > > extension/
> > > > > libload_this.so ->
> reference/lib/libtuscany_sca_reference.so
> > > > > xsd/
> > > > > ws-binding.xsd
> > > > > reference/
> > > > > lib/
> > > > > libtuscany_sca_reference.so
> > > > > service/
> > > > > lib/
> > > > > libtuscany_sca_service.so
> > > > >
> > > > > Here the runtime loads anything in any folder named 'extension' so
> > the
> > > > > reference extension would be loaded/initialized.
> > > >
> > > > Windows doesn't really do symlinks - I guess on windows we'd have
> the
> > > > tuscany_sca_reference.dll in extension/ and nothing in
> reference/lib/
> > > > or reference/bin/? For extensions that have dependencies on other
> > > > libraries, those dll's could go in lib/
> > >
> > >
> > > I would actually have a copy of the dll in extension on windows and
> > delete
> > > it to disable. For a large extension it would be very wasy to make a
> > tiny
> > > dll which just implements the initialise interface expected by the
> > runtime
> > > and leave everything else in a separate dll in bin/lib.
> > >
> > > > I would package the cpp language extension as
> > > > >
> > > > > cpp/
> > > > > bin/
> > > > > lib/
> > > > > include/
> > > > > xsd/
> > > > > extension/
> > > > >
> > > > > The bin, lib, include are exactly what you would expect from a
> > package
> > > > that
> > > > > you might want to build against. The extension/ folder would hold
> > the
> > > > > library that implements the extension interface and xsd holds...
> > well
> > > > xsds!
> > > > >
> > > >
> > > > All sounds good to me. +1
> > > >
> > > > Andy
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > Pete
> > >
> > > Ok, so I think I get it. My concern is that giving the developer the
> > flexibility to put "extension" and "xsd" anywhere they like means that
> > they
> > will accidentally duplicate these directories as these are pretty common
> > names for directories. So we could ask them always to have them at the
> top
> > level. Doesn't remove the possibility for duplication but reduces the
> risk
> > a
> > little. Or we could make the names a little more exotic with some
> > qualification "sca_extension", "sca_xsd" for example, and maintain the
> > flexibility you want.
> >
> > Simon
> >
> >
>
> Yes... the names are always the hardest thing to come up with! They should
> not be sca_ as this is Tuscany specific.
>
> so... we could get away with just the one folder tuscany_extension that
> contains the schema and libraries:
>
>
> cpp/
> bin/
> lib/
> include/
> xsd/
> tuscany_extension/ <- schema and libraries implementing extension i/f
> live under this tree
> xsd -> ../xsd <- could even be a symlink to where they really are
>
> I like that better. Runtime will load library and schema from any subtree
> under <sca_install>/extensions that is named 'tuscany_extension' and
> ignore
> anything else.
>
> Nice and flexible.
>
>
> --
> Pete
>
> That sounds like a plan. Nice one.
Simon
Re: [C++] Binding/Language Extensions - packaging
Posted by Andrew Borley <aj...@gmail.com>.
On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> >
> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >
> > > On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
> > > >
> > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > >
> > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > >
> > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > > > > >
> > > > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com>
> > wrote:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
> > > jsdelfino@apache.org>
> > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Pete Robbins wrote:
> > > > > > > > > > > > > Our current method of packaging and loading an
> > > extension
> > > > is
> > > > > > > > > fairly
> > > > > > > > > > > > > simple:
> > > > > > > > > > > > > we load all schema and libraries in the extensions
> > > path.
> > > > > > This
> > > > > > > > > has a
> > > > > > > > > > > > > number
> > > > > > > > > > > > > of problems.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 1. An extension may consist of more than one library
> > > e.g
> > > > .
> > > > > > > > > > > > > libmy_extension.so
> > > > > > > > > > > > > and libmy_extension_utils.so. Our current loading
> > > scheme
> > > > > > will
> > > > > > > > > > attempt
> > > > > > > > > > > > to
> > > > > > > > > > > > > load both of these and may fail on the one that
> > > doesn't
> > > > > > > provide
> > > > > > > > > the
> > > > > > > > > > > > > extension initialize method. On MacOS the output of
> > > our
> > > > > > build
> > > > > > > > > > produces
> > > > > > > > > > > > a
> > > > > > > > > > > > > libx.dylib and a series of symlinks to this called
> > > > > > > libx.0.dylib,
> > > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these
> > > which
> > > > > > > doesn't
> > > > > > > > > > cause
> > > > > > > > > > > > > problems as they are all the sam library and just
> > > > repeatedly
> > > > > > > > > > register
> > > > > > > > > > > > to
> > > > > > > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2. Control of whether or not to load an extension
> > > > library is
> > > > > > > > > > currently
> > > > > > > > > > > > by
> > > > > > > > > > > > > renaming the library so the runtime doesn't find it.
> > > An
> > > > > > > example
> > > > > > > > > is
> > > > > > > > > > > > > that we
> > > > > > > > > > > > > ship our python extension as
> > > > > > libtuscany_sca_python.so.diabled.
> > > > > > > > > This
> > > > > > > > > > is
> > > > > > > > > > > >
> > > > > > > > > > > > > horrible and error prone.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We could improve this by having a system
> > configuration
> > > > file
> > > > > > > that
> > > > > > > > > > lists
> > > > > > > > > > > > > the
> > > > > > > > > > > > > required extensions but the I like the self
> > contained
> > > > > > package
> > > > > > > > > > approach
> > > > > > > > > > > > > that
> > > > > > > > > > > > > we have now. I'd like to implement an improved
> > scheme
> > > > for
> > > > > > > > > packaging
> > > > > > > > > > an
> > > > > > > > > > > > > extension by introducing a per extension
> > configuration
> > > > file.
> > > > > > > > > > Something
> > > > > > > > > > > > > like:
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > > > > > > </scacppp:extension>
> > > > > > > > > > > > >
> > > > > > > > > > > > > So the package would look like:
> > > > > > > > > > > > >
> > > > > > > > > > > > > extensions/
> > > > > > > > > > > > > ws/
> > > > > > > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > > > > > > lib/
> > > > > > > > > > > > > xsd/
> > > > > > > > > > > > > other_folder/
> > > > > > > > > > > > > ...
> > > > > > > > > > > > >
> > > > > > > > > > > > > The .extension configuration file is saying to load
> > > the
> > > > > > > library
> > > > > > > > > > which
> > > > > > > > > > > > is
> > > > > > > > > > > > > located somewhere in the package... the runtime will
> > > > find
> > > > > > > it...
> > > > > > > > > no
> > > > > > > > > > > > > need to
> > > > > > > > > > > > > specify a path.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Taking this further the configuration file could
> > list
> > > > the
> > > > > > > schema
> > > > > > > > > to
> > > > > > > > > > be
> > > > > > > > > > > > > loaded. Currently the runtime will just load any it
> > > > finds
> > > > > > but
> > > > > > > > > these
> > > > > > > > > > > > > may not
> > > > > > > > > > > > > be needed by the runtime e.g. the schema may be for
> > > some
> > > > > > > > > extension
> > > > > > > > > > > > > implementation specific purpose.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think it would also be good for the extension
> > > > > > > initialization()
> > > > > > > > > > > > > method to
> > > > > > > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > > > > > > extension("/tuscany/extensions/ws"). This would
> > allow
> > > > the
> > > > > > > > > extension
> > > > > > > > > > > > > package
> > > > > > > > > > > > > to contain any configuration information that it
> > > needs.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd like to start by at least introducing the
> > > .extension
> > > > > > file
> > > > > > > > > for
> > > > > > > > > > each
> > > > > > > > > > > > > extension and loading only the specified
> > library(ies)
> > > if
> > > > the
> > > > > > > > > > extension
> > > > > > > > > > > > is
> > > > > > > > > > > > > enabled.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Two thoughts:
> > > > > > > > > > > > - convention over configuration
> > > > > > > > > > > > - the runtime should be consumable without having to
> > go
> > > > tweak
> > > > > > > XML
> > > > > > > > > > > > configuration files
> > > > > > > > > > > >
> > > > > > > > > > > > If I remember correctly, renaming the dlls to
> > .disabled
> > > > was a
> > > > > > > last
> > > > > > > > >
> > > > > > > > > > > > minute change to work around DLL loading errors with
> > our
> > > > M2
> > > > > > > > > release on
> > > > > > > > > > > > Windows.
> > > > > > > > > > > >
> > > > > > > > > > > > I agree that we should do better than renaming to
> > > > .disabled,
> > > > > > but
> > > > > > > > > I'd
> > > > > > > > > > > > like to understand better the actual issues that we
> > > faced
> > > > > > before
> > > > > > > > > > > > inventing yet another XML based runtime configuration
> > > > > > language
> > > > > > > :)
> > > > > > > > > > > >
> > > > > > > > > > > > I'm aware of the following issues:
> > > > > > > > > > > > 1. We need the runtime to load extension libs only,
> > not
> > > > other
> > > > > > > libs
> > > > > > > > > > under
> > > > > > > > > > > >
> > > > > > > > > > > > the extension directory which are not actual
> > extensions
> > > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that contribute
> > > to
> > > > > > SCDL,
> > > > > > > > > and
> > > > > > > > > > > > leave other XSDs under the extension directory alone
> > > > > > > > > > > > 3. Extensions that cannot be loaded because some of
> > > their
> > > > > > > > > dependencies
> > > > > > > > > > > > are not there should no break the runtime
> > > > > > > > > > > > 4. The system admin / installer should be able to
> > > disable
> > > > > > > > > extensions
> > > > > > > > > > > > that won't load because their dependencies are not
> > there
> > > > (I'm
> > > > > > > not
> > > > > > > > > yet
> > > > > > > > > > > > convinced that this is still an issue if we manage to
> > > > solve
> > > > > > > issue
> > > > > > > > > #3)
> > > > > > > > > > > >
> > > > > > > > > > > > Did we run into any other big issues?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > No, that's about it.
> > > > > > > > > > >
> > > > > > > > > > > 2. XSD loading can be done by convention (schemas for
> > the
> > > > > > runtime
> > > > > > > > > are
> > > > > > > > > > > always in a folder called 'xsd'
> > > > > > > > > > > 1. Could be solved in a similar way by only loading
> > > > libraries in
> > > > > > a
> > > > > > > > > > folder
> > > > > > > > > > > called ???
> > > > > > > > > > > 3. Can be solved by just ignoring the load
> > errors/issuing
> > > a
> > > > > > > warning
> > > > > > > > > > rather
> > > > > > > > > > > than giving up
> > > > > > > > > > > 4. Can be solved by solution to 3.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I recall now why 3. was a big problem. Windows sometimes
> > > > throws up
> > > > > > > an
> > > > > > > > > > error
> > > > > > > > > > dialog when the load fails so it was not just a case of
> > the
> > > > > > runtime
> > > > > > > > > > handling
> > > > > > > > > > the load failure so we had to "disable" the extension.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Pete
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Pete
> > > > > > > > > >
> > > > > > > > > > Pete, just summarizing so that I understand. It seems
> > there
> > > > are
> > > > > > two
> > > > > > > > > parts
> > > > > > > > > to this.
> > > > > > > > >
> > > > > > > > > A/ Building and installing an extension in order that it can
> > > be
> > > > > > > consumed
> > > > > > > > > by
> > > > > > > > > the runtime. Building and installing extensions is
> > potentially
> > > a
> > > > > > > > > separate
> > > > > > > > > activity from building and installing the runtime itself.
> > > > Looking at
> > > > > > > the
> > > > > > > > > M2
> > > > > > > > > download they all come together at the moment.
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes. It just happens that we are the only providers of
> > > extensions
> > > > at
> > > > > > the
> > > > > > > > moment.
> > > > > > > >
> > > > > > > > B/ Optionally consuming/enabling an extension in the runtime,
> > > once
> > > > the
> > > > > > > > > extension has been installed, in such a way that the runtime
> > > is
> > > > able
> > > > > > > to
> > > > > > > > > find
> > > > > > > > > and load it.
> > > > > > > >
> > > > > > > >
> > > > > > > > Yes.
> > > > > > > >
> > > > > > > > If I understand the previous posts (and the current code) A/
> > is
> > > > > > achieved
> > > > > > > > > by
> > > > > > > > > building the extension in the deployed folder layout
> > > > > > > > > extension type/
> > > > > > > > > bin
> > > > > > > > > lib
> > > > > > > > > xsd
> > > > > > > >
> > > > > > > >
> > > > > > > > The current runtime does not mandate any layout. It will seek
> > > and
> > > > > > > > destroy.. I mean seek and load any library it finds and any
> > .xsd
> > > > it
> > > > > > > finds.
> > > > > > > > That layout is just how we package extensions in ou build at
> > the
> > > > > > moment.
> > > > > > > >
> > > > > > > >
> > > > > > > > and Pete you are proposing that this is adjusted so that there
> > > are
> > > > > > > > > directories that hold just the stuff that the contains the
> > > > extension
> > > > > > > > > rather
> > > > > > > > > than the things it depends on.
> > > > > > > > > extension type/
> > > > > > > > > bin
> > > > > > > > > extensionbin?
> > > > > > > > > lib
> > > > > > > > > xsd
> > > > > > > > > extensionxsd?
> > > > > > > > >
> > > > > > > > > You are also proposing that B/ is achieved by ensuring that
> > > this
> > > > > > > > > directory
> > > > > > > > > structure be placed in a location that the the runtime can
> > > > search
> > > > > > for
> > > > > > > > > active
> > > > > > > > > extensions (as it does at the moment in the deploy
> > directory)
> > > > > > > >
> > > > > > > >
> > > > > > > > it currently loads any extensions from under the
> > > > > > > > <sca_install_dir>/extensions folder
> > > > > > > >
> > > > > > > > and that the
> > > > > > > > > runtime ignores any badly configured extensions if possible.
> > > > > > Removing
> > > > > > > an
> > > > > > > > > extension's directory from the deploy
> > > > > > > >
> > > > > > > >
> > > > > > > > extensions not deploy
> > > > > > > >
> > > > > > > > directory structure has the effect of
> > > > > > > > > disabling it so I guess this is the fallback if the runtime
> > > > can't
> > > > > > > > > continue.
> > > > > > > >
> > > > > > > >
> > > > > > > > That is true in today's implementation.
> > > > > > > >
> > > > > > > > Anything more complex and, as Jean-Sebastien suggests, you
> > start
> > > > > > getting
> > > > > > > > > into the full blown package and dependency management
> > problem
> > > > that
> > > > > > > many
> > > > > > > > > other systems try to solve in different ways, e.g. rpm or
> > > pear.
> > > > > > > >
> > > > > > > >
> > > > > > > > Agreed... I'll give up on that plan ;-)
> > > > > > > >
> > > > > > > >
> > > > > > > > What we need is to enable the runtime to identify which
> > > extension
> > > > > > > > libraries to load and which schema to load. My suggestion is:
> > > > > > > >
> > > > > > > > my_extension/
> > > > > > > > bin/
> > > > > > > > extension/
> > > > > > > > library_that_will_be_loaded_by_the_runtime.so
> > > > > > > > lib/
> > > > > > > > lib_my_extension.so
> > > > > > > > include/
> > > > > > > > ... some headers maybe
> > > > > > > > xsd/
> > > > > > > > my_binding.xsd
> > > > > > > > any_other_folder/
> > > > > > > > any_stuff_I_like.xsd
> > > > > > > >
> > > > > > > > So the runtime will load any xsd that is in a folder named xsd
> > > and
> > > > > > > attempt
> > > > > > > > to load any library in a folder called extension. It would not
> > > > attempt
> > > > > > > to
> > > > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > > > > > > >
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > --
> > > > > > > > Pete
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Pete
> > > > > > >
> > > > > > > Sounds like the right idea to me. What goes in the bin dir?.
> > > > >
> > > > >
> > > > >
> > > > > Anything you like!
> > > >
> > > >
> > > > On windows, .exe's and .dll's are often found in bin, whereas the
> > > > .lib's (used for linking against at compile time) are generally found
> > > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which
> > > > is more unix-like, and, IMO, nicer)
> > > >
> > > > >
> > > > > Should the
> > > > > > lib be...
> > > > > >
> > > > > > lib/
> > > > > > lib_my_extension.lib
> > > > >
> > > > >
> > > > > lif you like... again... anything you like.
> > > > >
> > > > > Maybe you could stick with bin,include, lib,xsd and then have your
> > > > "other"
> > > > > > directory have everything else in it that the extension might rely
> > > on.
> > > > > > Depends what you now anticipate being in bin and lib over and
> > above
> > > > the
> > > > > > actual library exposing the extension.
> > > > > >
> > > > > > Will you continue to mandate a specific place where extensions are
> > > > > > installed, i.e. from you post <sca_install_dir>/extensions folder?
> > > > >
> > > > >
> > > > > Yes
> > > > >
> > > > > So that
> > > > > > you can move an extension out of there if you don't want it to
> > load.
> > > > >
> > > > >
> > > > > Well the idea here is you don't need to remove the whole
> > package..just
> > > > the
> > > > > library from the extension dir
> > > > >
> > > > > Simon
> > > > > >
> > > > > >
> > > > > The layout of the extension package is whatever the package creater
> > > > wants to
> > > > > make it. We don't care. All we will care about is loading any
> > library
> > > in
> > > > a
> > > > > folder anywhere in the extension package that is in a folder called
> > > > > 'extension' and expecting it to implement the extension initialize
> > > > method.
> > > > > We will also load any schema files in any folder called 'xsd'. An
> > > > extension
> > > > > only needs the xsd folder if it has schema required for the runtime
> > > > model.
> > > > >
> > > > > As an example here is the ws binding extension implemented for
> > axis2c:
> > > > >
> > > > > <sca_install>/extensions/
> > > > > ... somewhere in this tree... could be anywhere...
> > > > > ws/
> > > > > xsd/
> > > > > ws-binding.xsd
> > > > > reference/
> > > > > lib/
> > > > > libtuscany_sca_reference.so
> > > > > service/
> > > > > lib/
> > > > > libtuscany_sca_service.so
> > > > >
> > > > >
> > > > > OK... today the runtime would load the ws-binding.xsd as it loads
> > any
> > > > xsd it
> > > > > finds. It would also load any .so it finds so both reference aand
> > > > service
> > > > > extensions would be loaded/initialized.
> > > > >
> > > > > With my proposed change the xsd would be loaded (as it's in a folder
> > > > named
> > > > > 'xsd' ) but the libraries would not. To get the extensions loaded
> > you
> > > > would
> > > > > have to package with an 'extension' folder ... again anywhere in the
> > > > tree...
> > > > > which "contains" the libraries to load eg.
> > > > >
> > > > > ws/
> > > > > extension/
> > > > > libload_this.so -> reference/lib/libtuscany_sca_reference.so
> > > > > xsd/
> > > > > ws-binding.xsd
> > > > > reference/
> > > > > lib/
> > > > > libtuscany_sca_reference.so
> > > > > service/
> > > > > lib/
> > > > > libtuscany_sca_service.so
> > > > >
> > > > > Here the runtime loads anything in any folder named 'extension' so
> > the
> > > > > reference extension would be loaded/initialized.
> > > >
> > > > Windows doesn't really do symlinks - I guess on windows we'd have the
> > > > tuscany_sca_reference.dll in extension/ and nothing in reference/lib/
> > > > or reference/bin/? For extensions that have dependencies on other
> > > > libraries, those dll's could go in lib/
> > >
> > >
> > > I would actually have a copy of the dll in extension on windows and
> > delete
> > > it to disable. For a large extension it would be very wasy to make a
> > tiny
> > > dll which just implements the initialise interface expected by the
> > runtime
> > > and leave everything else in a separate dll in bin/lib.
> > >
> > > > I would package the cpp language extension as
> > > > >
> > > > > cpp/
> > > > > bin/
> > > > > lib/
> > > > > include/
> > > > > xsd/
> > > > > extension/
> > > > >
> > > > > The bin, lib, include are exactly what you would expect from a
> > package
> > > > that
> > > > > you might want to build against. The extension/ folder would hold
> > the
> > > > > library that implements the extension interface and xsd holds...
> > well
> > > > xsds!
> > > > >
> > > >
> > > > All sounds good to me. +1
> > > >
> > > > Andy
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > Pete
> > >
> > > Ok, so I think I get it. My concern is that giving the developer the
> > flexibility to put "extension" and "xsd" anywhere they like means that
> > they
> > will accidentally duplicate these directories as these are pretty common
> > names for directories. So we could ask them always to have them at the top
> > level. Doesn't remove the possibility for duplication but reduces the risk
> > a
> > little. Or we could make the names a little more exotic with some
> > qualification "sca_extension", "sca_xsd" for example, and maintain the
> > flexibility you want.
> >
> > Simon
> >
> >
>
> Yes... the names are always the hardest thing to come up with! They should
> not be sca_ as this is Tuscany specific.
>
> so... we could get away with just the one folder tuscany_extension that
> contains the schema and libraries:
>
>
> cpp/
> bin/
> lib/
> include/
> xsd/
> tuscany_extension/ <- schema and libraries implementing extension i/f
> live under this tree
> xsd -> ../xsd <- could even be a symlink to where they really are
>
> I like that better. Runtime will load library and schema from any subtree
> under <sca_install>/extensions that is named 'tuscany_extension' and ignore
> anything else.
>
> Nice and flexible.
Works for me. +1
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
>
> On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >
> > On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
> > >
> > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > >
> > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > >
> > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > > > >
> > > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > > >
> > > > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com>
> wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
> > jsdelfino@apache.org>
> > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Pete Robbins wrote:
> > > > > > > > > > > > Our current method of packaging and loading an
> > extension
> > > is
> > > > > > > > fairly
> > > > > > > > > > > > simple:
> > > > > > > > > > > > we load all schema and libraries in the extensions
> > path.
> > > > > This
> > > > > > > > has a
> > > > > > > > > > > > number
> > > > > > > > > > > > of problems.
> > > > > > > > > > > >
> > > > > > > > > > > > 1. An extension may consist of more than one library
> > e.g
> > > .
> > > > > > > > > > > > libmy_extension.so
> > > > > > > > > > > > and libmy_extension_utils.so. Our current loading
> > scheme
> > > > > will
> > > > > > > > > attempt
> > > > > > > > > > > to
> > > > > > > > > > > > load both of these and may fail on the one that
> > doesn't
> > > > > > provide
> > > > > > > > the
> > > > > > > > > > > > extension initialize method. On MacOS the output of
> > our
> > > > > build
> > > > > > > > > produces
> > > > > > > > > > > a
> > > > > > > > > > > > libx.dylib and a series of symlinks to this called
> > > > > > libx.0.dylib,
> > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these
> > which
> > > > > > doesn't
> > > > > > > > > cause
> > > > > > > > > > > > problems as they are all the sam library and just
> > > repeatedly
> > > > > > > > > register
> > > > > > > > > > > to
> > > > > > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > > > > > >
> > > > > > > > > > > > 2. Control of whether or not to load an extension
> > > library is
> > > > > > > > > currently
> > > > > > > > > > > by
> > > > > > > > > > > > renaming the library so the runtime doesn't find it.
> > An
> > > > > > example
> > > > > > > > is
> > > > > > > > > > > > that we
> > > > > > > > > > > > ship our python extension as
> > > > > libtuscany_sca_python.so.diabled.
> > > > > > > > This
> > > > > > > > > is
> > > > > > > > > > >
> > > > > > > > > > > > horrible and error prone.
> > > > > > > > > > > >
> > > > > > > > > > > > We could improve this by having a system
> configuration
> > > file
> > > > > > that
> > > > > > > > > lists
> > > > > > > > > > > > the
> > > > > > > > > > > > required extensions but the I like the self
> contained
> > > > > package
> > > > > > > > > approach
> > > > > > > > > > > > that
> > > > > > > > > > > > we have now. I'd like to implement an improved
> scheme
> > > for
> > > > > > > > packaging
> > > > > > > > > an
> > > > > > > > > > > > extension by introducing a per extension
> configuration
> > > file.
> > > > > > > > > Something
> > > > > > > > > > > > like:
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > > > > > </scacppp:extension>
> > > > > > > > > > > >
> > > > > > > > > > > > So the package would look like:
> > > > > > > > > > > >
> > > > > > > > > > > > extensions/
> > > > > > > > > > > > ws/
> > > > > > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > > > > > lib/
> > > > > > > > > > > > xsd/
> > > > > > > > > > > > other_folder/
> > > > > > > > > > > > ...
> > > > > > > > > > > >
> > > > > > > > > > > > The .extension configuration file is saying to load
> > the
> > > > > > library
> > > > > > > > > which
> > > > > > > > > > > is
> > > > > > > > > > > > located somewhere in the package... the runtime will
> > > find
> > > > > > it...
> > > > > > > > no
> > > > > > > > > > > > need to
> > > > > > > > > > > > specify a path.
> > > > > > > > > > > >
> > > > > > > > > > > > Taking this further the configuration file could
> list
> > > the
> > > > > > schema
> > > > > > > > to
> > > > > > > > > be
> > > > > > > > > > > > loaded. Currently the runtime will just load any it
> > > finds
> > > > > but
> > > > > > > > these
> > > > > > > > > > > > may not
> > > > > > > > > > > > be needed by the runtime e.g. the schema may be for
> > some
> > > > > > > > extension
> > > > > > > > > > > > implementation specific purpose.
> > > > > > > > > > > >
> > > > > > > > > > > > I think it would also be good for the extension
> > > > > > initialization()
> > > > > > > > > > > > method to
> > > > > > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > > > > > extension("/tuscany/extensions/ws"). This would
> allow
> > > the
> > > > > > > > extension
> > > > > > > > > > > > package
> > > > > > > > > > > > to contain any configuration information that it
> > needs.
> > > > > > > > > > > >
> > > > > > > > > > > > I'd like to start by at least introducing the
> > .extension
> > > > > file
> > > > > > > > for
> > > > > > > > > each
> > > > > > > > > > > > extension and loading only the specified
> library(ies)
> > if
> > > the
> > > > > > > > > extension
> > > > > > > > > > > is
> > > > > > > > > > > > enabled.
> > > > > > > > > > > >
> > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Two thoughts:
> > > > > > > > > > > - convention over configuration
> > > > > > > > > > > - the runtime should be consumable without having to
> go
> > > tweak
> > > > > > XML
> > > > > > > > > > > configuration files
> > > > > > > > > > >
> > > > > > > > > > > If I remember correctly, renaming the dlls to
> .disabled
> > > was a
> > > > > > last
> > > > > > > >
> > > > > > > > > > > minute change to work around DLL loading errors with
> our
> > > M2
> > > > > > > > release on
> > > > > > > > > > > Windows.
> > > > > > > > > > >
> > > > > > > > > > > I agree that we should do better than renaming to
> > > .disabled,
> > > > > but
> > > > > > > > I'd
> > > > > > > > > > > like to understand better the actual issues that we
> > faced
> > > > > before
> > > > > > > > > > > inventing yet another XML based runtime configuration
> > > > > language
> > > > > > :)
> > > > > > > > > > >
> > > > > > > > > > > I'm aware of the following issues:
> > > > > > > > > > > 1. We need the runtime to load extension libs only,
> not
> > > other
> > > > > > libs
> > > > > > > > > under
> > > > > > > > > > >
> > > > > > > > > > > the extension directory which are not actual
> extensions
> > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that contribute
> > to
> > > > > SCDL,
> > > > > > > > and
> > > > > > > > > > > leave other XSDs under the extension directory alone
> > > > > > > > > > > 3. Extensions that cannot be loaded because some of
> > their
> > > > > > > > dependencies
> > > > > > > > > > > are not there should no break the runtime
> > > > > > > > > > > 4. The system admin / installer should be able to
> > disable
> > > > > > > > extensions
> > > > > > > > > > > that won't load because their dependencies are not
> there
> > > (I'm
> > > > > > not
> > > > > > > > yet
> > > > > > > > > > > convinced that this is still an issue if we manage to
> > > solve
> > > > > > issue
> > > > > > > > #3)
> > > > > > > > > > >
> > > > > > > > > > > Did we run into any other big issues?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > No, that's about it.
> > > > > > > > > >
> > > > > > > > > > 2. XSD loading can be done by convention (schemas for
> the
> > > > > runtime
> > > > > > > > are
> > > > > > > > > > always in a folder called 'xsd'
> > > > > > > > > > 1. Could be solved in a similar way by only loading
> > > libraries in
> > > > > a
> > > > > > > > > folder
> > > > > > > > > > called ???
> > > > > > > > > > 3. Can be solved by just ignoring the load
> errors/issuing
> > a
> > > > > > warning
> > > > > > > > > rather
> > > > > > > > > > than giving up
> > > > > > > > > > 4. Can be solved by solution to 3.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I recall now why 3. was a big problem. Windows sometimes
> > > throws up
> > > > > > an
> > > > > > > > > error
> > > > > > > > > dialog when the load fails so it was not just a case of
> the
> > > > > runtime
> > > > > > > > > handling
> > > > > > > > > the load failure so we had to "disable" the extension.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Pete
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Pete
> > > > > > > > >
> > > > > > > > > Pete, just summarizing so that I understand. It seems
> there
> > > are
> > > > > two
> > > > > > > > parts
> > > > > > > > to this.
> > > > > > > >
> > > > > > > > A/ Building and installing an extension in order that it can
> > be
> > > > > > consumed
> > > > > > > > by
> > > > > > > > the runtime. Building and installing extensions is
> potentially
> > a
> > > > > > > > separate
> > > > > > > > activity from building and installing the runtime itself.
> > > Looking at
> > > > > > the
> > > > > > > > M2
> > > > > > > > download they all come together at the moment.
> > > > > > >
> > > > > > >
> > > > > > > Yes. It just happens that we are the only providers of
> > extensions
> > > at
> > > > > the
> > > > > > > moment.
> > > > > > >
> > > > > > > B/ Optionally consuming/enabling an extension in the runtime,
> > once
> > > the
> > > > > > > > extension has been installed, in such a way that the runtime
> > is
> > > able
> > > > > > to
> > > > > > > > find
> > > > > > > > and load it.
> > > > > > >
> > > > > > >
> > > > > > > Yes.
> > > > > > >
> > > > > > > If I understand the previous posts (and the current code) A/
> is
> > > > > achieved
> > > > > > > > by
> > > > > > > > building the extension in the deployed folder layout
> > > > > > > > extension type/
> > > > > > > > bin
> > > > > > > > lib
> > > > > > > > xsd
> > > > > > >
> > > > > > >
> > > > > > > The current runtime does not mandate any layout. It will seek
> > and
> > > > > > > destroy.. I mean seek and load any library it finds and any
> .xsd
> > > it
> > > > > > finds.
> > > > > > > That layout is just how we package extensions in ou build at
> the
> > > > > moment.
> > > > > > >
> > > > > > >
> > > > > > > and Pete you are proposing that this is adjusted so that there
> > are
> > > > > > > > directories that hold just the stuff that the contains the
> > > extension
> > > > > > > > rather
> > > > > > > > than the things it depends on.
> > > > > > > > extension type/
> > > > > > > > bin
> > > > > > > > extensionbin?
> > > > > > > > lib
> > > > > > > > xsd
> > > > > > > > extensionxsd?
> > > > > > > >
> > > > > > > > You are also proposing that B/ is achieved by ensuring that
> > this
> > > > > > > > directory
> > > > > > > > structure be placed in a location that the the runtime can
> > > search
> > > > > for
> > > > > > > > active
> > > > > > > > extensions (as it does at the moment in the deploy
> directory)
> > > > > > >
> > > > > > >
> > > > > > > it currently loads any extensions from under the
> > > > > > > <sca_install_dir>/extensions folder
> > > > > > >
> > > > > > > and that the
> > > > > > > > runtime ignores any badly configured extensions if possible.
> > > > > Removing
> > > > > > an
> > > > > > > > extension's directory from the deploy
> > > > > > >
> > > > > > >
> > > > > > > extensions not deploy
> > > > > > >
> > > > > > > directory structure has the effect of
> > > > > > > > disabling it so I guess this is the fallback if the runtime
> > > can't
> > > > > > > > continue.
> > > > > > >
> > > > > > >
> > > > > > > That is true in today's implementation.
> > > > > > >
> > > > > > > Anything more complex and, as Jean-Sebastien suggests, you
> start
> > > > > getting
> > > > > > > > into the full blown package and dependency management
> problem
> > > that
> > > > > > many
> > > > > > > > other systems try to solve in different ways, e.g. rpm or
> > pear.
> > > > > > >
> > > > > > >
> > > > > > > Agreed... I'll give up on that plan ;-)
> > > > > > >
> > > > > > >
> > > > > > > What we need is to enable the runtime to identify which
> > extension
> > > > > > > libraries to load and which schema to load. My suggestion is:
> > > > > > >
> > > > > > > my_extension/
> > > > > > > bin/
> > > > > > > extension/
> > > > > > > library_that_will_be_loaded_by_the_runtime.so
> > > > > > > lib/
> > > > > > > lib_my_extension.so
> > > > > > > include/
> > > > > > > ... some headers maybe
> > > > > > > xsd/
> > > > > > > my_binding.xsd
> > > > > > > any_other_folder/
> > > > > > > any_stuff_I_like.xsd
> > > > > > >
> > > > > > > So the runtime will load any xsd that is in a folder named xsd
> > and
> > > > > > attempt
> > > > > > > to load any library in a folder called extension. It would not
> > > attempt
> > > > > > to
> > > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > > > > > >
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > --
> > > > > > > Pete
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Pete
> > > > > >
> > > > > > Sounds like the right idea to me. What goes in the bin dir?.
> > > >
> > > >
> > > >
> > > > Anything you like!
> > >
> > >
> > > On windows, .exe's and .dll's are often found in bin, whereas the
> > > .lib's (used for linking against at compile time) are generally found
> > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which
> > > is more unix-like, and, IMO, nicer)
> > >
> > > >
> > > > Should the
> > > > > lib be...
> > > > >
> > > > > lib/
> > > > > lib_my_extension.lib
> > > >
> > > >
> > > > lif you like... again... anything you like.
> > > >
> > > > Maybe you could stick with bin,include, lib,xsd and then have your
> > > "other"
> > > > > directory have everything else in it that the extension might rely
> > on.
> > > > > Depends what you now anticipate being in bin and lib over and
> above
> > > the
> > > > > actual library exposing the extension.
> > > > >
> > > > > Will you continue to mandate a specific place where extensions are
> > > > > installed, i.e. from you post <sca_install_dir>/extensions folder?
> > > >
> > > >
> > > > Yes
> > > >
> > > > So that
> > > > > you can move an extension out of there if you don't want it to
> load.
> > > >
> > > >
> > > > Well the idea here is you don't need to remove the whole
> package..just
> > > the
> > > > library from the extension dir
> > > >
> > > > Simon
> > > > >
> > > > >
> > > > The layout of the extension package is whatever the package creater
> > > wants to
> > > > make it. We don't care. All we will care about is loading any
> library
> > in
> > > a
> > > > folder anywhere in the extension package that is in a folder called
> > > > 'extension' and expecting it to implement the extension initialize
> > > method.
> > > > We will also load any schema files in any folder called 'xsd'. An
> > > extension
> > > > only needs the xsd folder if it has schema required for the runtime
> > > model.
> > > >
> > > > As an example here is the ws binding extension implemented for
> axis2c:
> > > >
> > > > <sca_install>/extensions/
> > > > ... somewhere in this tree... could be anywhere...
> > > > ws/
> > > > xsd/
> > > > ws-binding.xsd
> > > > reference/
> > > > lib/
> > > > libtuscany_sca_reference.so
> > > > service/
> > > > lib/
> > > > libtuscany_sca_service.so
> > > >
> > > >
> > > > OK... today the runtime would load the ws-binding.xsd as it loads
> any
> > > xsd it
> > > > finds. It would also load any .so it finds so both reference aand
> > > service
> > > > extensions would be loaded/initialized.
> > > >
> > > > With my proposed change the xsd would be loaded (as it's in a folder
> > > named
> > > > 'xsd' ) but the libraries would not. To get the extensions loaded
> you
> > > would
> > > > have to package with an 'extension' folder ... again anywhere in the
> > > tree...
> > > > which "contains" the libraries to load eg.
> > > >
> > > > ws/
> > > > extension/
> > > > libload_this.so -> reference/lib/libtuscany_sca_reference.so
> > > > xsd/
> > > > ws-binding.xsd
> > > > reference/
> > > > lib/
> > > > libtuscany_sca_reference.so
> > > > service/
> > > > lib/
> > > > libtuscany_sca_service.so
> > > >
> > > > Here the runtime loads anything in any folder named 'extension' so
> the
> > > > reference extension would be loaded/initialized.
> > >
> > > Windows doesn't really do symlinks - I guess on windows we'd have the
> > > tuscany_sca_reference.dll in extension/ and nothing in reference/lib/
> > > or reference/bin/? For extensions that have dependencies on other
> > > libraries, those dll's could go in lib/
> >
> >
> > I would actually have a copy of the dll in extension on windows and
> delete
> > it to disable. For a large extension it would be very wasy to make a
> tiny
> > dll which just implements the initialise interface expected by the
> runtime
> > and leave everything else in a separate dll in bin/lib.
> >
> > > I would package the cpp language extension as
> > > >
> > > > cpp/
> > > > bin/
> > > > lib/
> > > > include/
> > > > xsd/
> > > > extension/
> > > >
> > > > The bin, lib, include are exactly what you would expect from a
> package
> > > that
> > > > you might want to build against. The extension/ folder would hold
> the
> > > > library that implements the extension interface and xsd holds...
> well
> > > xsds!
> > > >
> > >
> > > All sounds good to me. +1
> > >
> > > Andy
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> > >
> > >
> >
> >
> > --
> > Pete
> >
> > Ok, so I think I get it. My concern is that giving the developer the
> flexibility to put "extension" and "xsd" anywhere they like means that
> they
> will accidentally duplicate these directories as these are pretty common
> names for directories. So we could ask them always to have them at the top
> level. Doesn't remove the possibility for duplication but reduces the risk
> a
> little. Or we could make the names a little more exotic with some
> qualification "sca_extension", "sca_xsd" for example, and maintain the
> flexibility you want.
>
> Simon
>
>
Yes... the names are always the hardest thing to come up with! They should
not be sca_ as this is Tuscany specific.
so... we could get away with just the one folder tuscany_extension that
contains the schema and libraries:
cpp/
bin/
lib/
include/
xsd/
tuscany_extension/ <- schema and libraries implementing extension i/f
live under this tree
xsd -> ../xsd <- could even be a symlink to where they really are
I like that better. Runtime will load library and schema from any subtree
under <sca_install>/extensions that is named 'tuscany_extension' and ignore
anything else.
Nice and flexible.
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Simon Laws <si...@googlemail.com>.
On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>
> On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
> >
> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > >
> > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > >
> > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > > >
> > > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > >
> > > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 30/11/06, Jean-Sebastien Delfino <
> jsdelfino@apache.org>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Pete Robbins wrote:
> > > > > > > > > > > Our current method of packaging and loading an
> extension
> > is
> > > > > > > fairly
> > > > > > > > > > > simple:
> > > > > > > > > > > we load all schema and libraries in the extensions
> path.
> > > > This
> > > > > > > has a
> > > > > > > > > > > number
> > > > > > > > > > > of problems.
> > > > > > > > > > >
> > > > > > > > > > > 1. An extension may consist of more than one library
> e.g
> > .
> > > > > > > > > > > libmy_extension.so
> > > > > > > > > > > and libmy_extension_utils.so. Our current loading
> scheme
> > > > will
> > > > > > > > attempt
> > > > > > > > > > to
> > > > > > > > > > > load both of these and may fail on the one that
> doesn't
> > > > > provide
> > > > > > > the
> > > > > > > > > > > extension initialize method. On MacOS the output of
> our
> > > > build
> > > > > > > > produces
> > > > > > > > > > a
> > > > > > > > > > > libx.dylib and a series of symlinks to this called
> > > > > libx.0.dylib,
> > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these
> which
> > > > > doesn't
> > > > > > > > cause
> > > > > > > > > > > problems as they are all the sam library and just
> > repeatedly
> > > > > > > > register
> > > > > > > > > > to
> > > > > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > > > > >
> > > > > > > > > > > 2. Control of whether or not to load an extension
> > library is
> > > > > > > > currently
> > > > > > > > > > by
> > > > > > > > > > > renaming the library so the runtime doesn't find it.
> An
> > > > > example
> > > > > > > is
> > > > > > > > > > > that we
> > > > > > > > > > > ship our python extension as
> > > > libtuscany_sca_python.so.diabled.
> > > > > > > This
> > > > > > > > is
> > > > > > > > > >
> > > > > > > > > > > horrible and error prone.
> > > > > > > > > > >
> > > > > > > > > > > We could improve this by having a system configuration
> > file
> > > > > that
> > > > > > > > lists
> > > > > > > > > > > the
> > > > > > > > > > > required extensions but the I like the self contained
> > > > package
> > > > > > > > approach
> > > > > > > > > > > that
> > > > > > > > > > > we have now. I'd like to implement an improved scheme
> > for
> > > > > > > packaging
> > > > > > > > an
> > > > > > > > > > > extension by introducing a per extension configuration
> > file.
> > > > > > > > Something
> > > > > > > > > > > like:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > > > > </scacppp:extension>
> > > > > > > > > > >
> > > > > > > > > > > So the package would look like:
> > > > > > > > > > >
> > > > > > > > > > > extensions/
> > > > > > > > > > > ws/
> > > > > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > > > > lib/
> > > > > > > > > > > xsd/
> > > > > > > > > > > other_folder/
> > > > > > > > > > > ...
> > > > > > > > > > >
> > > > > > > > > > > The .extension configuration file is saying to load
> the
> > > > > library
> > > > > > > > which
> > > > > > > > > > is
> > > > > > > > > > > located somewhere in the package... the runtime will
> > find
> > > > > it...
> > > > > > > no
> > > > > > > > > > > need to
> > > > > > > > > > > specify a path.
> > > > > > > > > > >
> > > > > > > > > > > Taking this further the configuration file could list
> > the
> > > > > schema
> > > > > > > to
> > > > > > > > be
> > > > > > > > > > > loaded. Currently the runtime will just load any it
> > finds
> > > > but
> > > > > > > these
> > > > > > > > > > > may not
> > > > > > > > > > > be needed by the runtime e.g. the schema may be for
> some
> > > > > > > extension
> > > > > > > > > > > implementation specific purpose.
> > > > > > > > > > >
> > > > > > > > > > > I think it would also be good for the extension
> > > > > initialization()
> > > > > > > > > > > method to
> > > > > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > > > > extension("/tuscany/extensions/ws"). This would allow
> > the
> > > > > > > extension
> > > > > > > > > > > package
> > > > > > > > > > > to contain any configuration information that it
> needs.
> > > > > > > > > > >
> > > > > > > > > > > I'd like to start by at least introducing the
> .extension
> > > > file
> > > > > > > for
> > > > > > > > each
> > > > > > > > > > > extension and loading only the specified library(ies)
> if
> > the
> > > > > > > > extension
> > > > > > > > > > is
> > > > > > > > > > > enabled.
> > > > > > > > > > >
> > > > > > > > > > > Any thoughts?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Two thoughts:
> > > > > > > > > > - convention over configuration
> > > > > > > > > > - the runtime should be consumable without having to go
> > tweak
> > > > > XML
> > > > > > > > > > configuration files
> > > > > > > > > >
> > > > > > > > > > If I remember correctly, renaming the dlls to .disabled
> > was a
> > > > > last
> > > > > > >
> > > > > > > > > > minute change to work around DLL loading errors with our
> > M2
> > > > > > > release on
> > > > > > > > > > Windows.
> > > > > > > > > >
> > > > > > > > > > I agree that we should do better than renaming to
> > .disabled,
> > > > but
> > > > > > > I'd
> > > > > > > > > > like to understand better the actual issues that we
> faced
> > > > before
> > > > > > > > > > inventing yet another XML based runtime configuration
> > > > language
> > > > > :)
> > > > > > > > > >
> > > > > > > > > > I'm aware of the following issues:
> > > > > > > > > > 1. We need the runtime to load extension libs only, not
> > other
> > > > > libs
> > > > > > > > under
> > > > > > > > > >
> > > > > > > > > > the extension directory which are not actual extensions
> > > > > > > > > > 2. Same for XSDs, we need to load XSDs that contribute
> to
> > > > SCDL,
> > > > > > > and
> > > > > > > > > > leave other XSDs under the extension directory alone
> > > > > > > > > > 3. Extensions that cannot be loaded because some of
> their
> > > > > > > dependencies
> > > > > > > > > > are not there should no break the runtime
> > > > > > > > > > 4. The system admin / installer should be able to
> disable
> > > > > > > extensions
> > > > > > > > > > that won't load because their dependencies are not there
> > (I'm
> > > > > not
> > > > > > > yet
> > > > > > > > > > convinced that this is still an issue if we manage to
> > solve
> > > > > issue
> > > > > > > #3)
> > > > > > > > > >
> > > > > > > > > > Did we run into any other big issues?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > No, that's about it.
> > > > > > > > >
> > > > > > > > > 2. XSD loading can be done by convention (schemas for the
> > > > runtime
> > > > > > > are
> > > > > > > > > always in a folder called 'xsd'
> > > > > > > > > 1. Could be solved in a similar way by only loading
> > libraries in
> > > > a
> > > > > > > > folder
> > > > > > > > > called ???
> > > > > > > > > 3. Can be solved by just ignoring the load errors/issuing
> a
> > > > > warning
> > > > > > > > rather
> > > > > > > > > than giving up
> > > > > > > > > 4. Can be solved by solution to 3.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > I recall now why 3. was a big problem. Windows sometimes
> > throws up
> > > > > an
> > > > > > > > error
> > > > > > > > dialog when the load fails so it was not just a case of the
> > > > runtime
> > > > > > > > handling
> > > > > > > > the load failure so we had to "disable" the extension.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > --
> > > > > > > > Pete
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Pete
> > > > > > > >
> > > > > > > > Pete, just summarizing so that I understand. It seems there
> > are
> > > > two
> > > > > > > parts
> > > > > > > to this.
> > > > > > >
> > > > > > > A/ Building and installing an extension in order that it can
> be
> > > > > consumed
> > > > > > > by
> > > > > > > the runtime. Building and installing extensions is potentially
> a
> > > > > > > separate
> > > > > > > activity from building and installing the runtime itself.
> > Looking at
> > > > > the
> > > > > > > M2
> > > > > > > download they all come together at the moment.
> > > > > >
> > > > > >
> > > > > > Yes. It just happens that we are the only providers of
> extensions
> > at
> > > > the
> > > > > > moment.
> > > > > >
> > > > > > B/ Optionally consuming/enabling an extension in the runtime,
> once
> > the
> > > > > > > extension has been installed, in such a way that the runtime
> is
> > able
> > > > > to
> > > > > > > find
> > > > > > > and load it.
> > > > > >
> > > > > >
> > > > > > Yes.
> > > > > >
> > > > > > If I understand the previous posts (and the current code) A/ is
> > > > achieved
> > > > > > > by
> > > > > > > building the extension in the deployed folder layout
> > > > > > > extension type/
> > > > > > > bin
> > > > > > > lib
> > > > > > > xsd
> > > > > >
> > > > > >
> > > > > > The current runtime does not mandate any layout. It will seek
> and
> > > > > > destroy.. I mean seek and load any library it finds and any .xsd
> > it
> > > > > finds.
> > > > > > That layout is just how we package extensions in ou build at the
> > > > moment.
> > > > > >
> > > > > >
> > > > > > and Pete you are proposing that this is adjusted so that there
> are
> > > > > > > directories that hold just the stuff that the contains the
> > extension
> > > > > > > rather
> > > > > > > than the things it depends on.
> > > > > > > extension type/
> > > > > > > bin
> > > > > > > extensionbin?
> > > > > > > lib
> > > > > > > xsd
> > > > > > > extensionxsd?
> > > > > > >
> > > > > > > You are also proposing that B/ is achieved by ensuring that
> this
> > > > > > > directory
> > > > > > > structure be placed in a location that the the runtime can
> > search
> > > > for
> > > > > > > active
> > > > > > > extensions (as it does at the moment in the deploy directory)
> > > > > >
> > > > > >
> > > > > > it currently loads any extensions from under the
> > > > > > <sca_install_dir>/extensions folder
> > > > > >
> > > > > > and that the
> > > > > > > runtime ignores any badly configured extensions if possible.
> > > > Removing
> > > > > an
> > > > > > > extension's directory from the deploy
> > > > > >
> > > > > >
> > > > > > extensions not deploy
> > > > > >
> > > > > > directory structure has the effect of
> > > > > > > disabling it so I guess this is the fallback if the runtime
> > can't
> > > > > > > continue.
> > > > > >
> > > > > >
> > > > > > That is true in today's implementation.
> > > > > >
> > > > > > Anything more complex and, as Jean-Sebastien suggests, you start
> > > > getting
> > > > > > > into the full blown package and dependency management problem
> > that
> > > > > many
> > > > > > > other systems try to solve in different ways, e.g. rpm or
> pear.
> > > > > >
> > > > > >
> > > > > > Agreed... I'll give up on that plan ;-)
> > > > > >
> > > > > >
> > > > > > What we need is to enable the runtime to identify which
> extension
> > > > > > libraries to load and which schema to load. My suggestion is:
> > > > > >
> > > > > > my_extension/
> > > > > > bin/
> > > > > > extension/
> > > > > > library_that_will_be_loaded_by_the_runtime.so
> > > > > > lib/
> > > > > > lib_my_extension.so
> > > > > > include/
> > > > > > ... some headers maybe
> > > > > > xsd/
> > > > > > my_binding.xsd
> > > > > > any_other_folder/
> > > > > > any_stuff_I_like.xsd
> > > > > >
> > > > > > So the runtime will load any xsd that is in a folder named xsd
> and
> > > > > attempt
> > > > > > to load any library in a folder called extension. It would not
> > attempt
> > > > > to
> > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > --
> > > > > > Pete
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pete
> > > > >
> > > > > Sounds like the right idea to me. What goes in the bin dir?.
> > >
> > >
> > >
> > > Anything you like!
> >
> >
> > On windows, .exe's and .dll's are often found in bin, whereas the
> > .lib's (used for linking against at compile time) are generally found
> > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which
> > is more unix-like, and, IMO, nicer)
> >
> > >
> > > Should the
> > > > lib be...
> > > >
> > > > lib/
> > > > lib_my_extension.lib
> > >
> > >
> > > lif you like... again... anything you like.
> > >
> > > Maybe you could stick with bin,include, lib,xsd and then have your
> > "other"
> > > > directory have everything else in it that the extension might rely
> on.
> > > > Depends what you now anticipate being in bin and lib over and above
> > the
> > > > actual library exposing the extension.
> > > >
> > > > Will you continue to mandate a specific place where extensions are
> > > > installed, i.e. from you post <sca_install_dir>/extensions folder?
> > >
> > >
> > > Yes
> > >
> > > So that
> > > > you can move an extension out of there if you don't want it to load.
> > >
> > >
> > > Well the idea here is you don't need to remove the whole package..just
> > the
> > > library from the extension dir
> > >
> > > Simon
> > > >
> > > >
> > > The layout of the extension package is whatever the package creater
> > wants to
> > > make it. We don't care. All we will care about is loading any library
> in
> > a
> > > folder anywhere in the extension package that is in a folder called
> > > 'extension' and expecting it to implement the extension initialize
> > method.
> > > We will also load any schema files in any folder called 'xsd'. An
> > extension
> > > only needs the xsd folder if it has schema required for the runtime
> > model.
> > >
> > > As an example here is the ws binding extension implemented for axis2c:
> > >
> > > <sca_install>/extensions/
> > > ... somewhere in this tree... could be anywhere...
> > > ws/
> > > xsd/
> > > ws-binding.xsd
> > > reference/
> > > lib/
> > > libtuscany_sca_reference.so
> > > service/
> > > lib/
> > > libtuscany_sca_service.so
> > >
> > >
> > > OK... today the runtime would load the ws-binding.xsd as it loads any
> > xsd it
> > > finds. It would also load any .so it finds so both reference aand
> > service
> > > extensions would be loaded/initialized.
> > >
> > > With my proposed change the xsd would be loaded (as it's in a folder
> > named
> > > 'xsd' ) but the libraries would not. To get the extensions loaded you
> > would
> > > have to package with an 'extension' folder ... again anywhere in the
> > tree...
> > > which "contains" the libraries to load eg.
> > >
> > > ws/
> > > extension/
> > > libload_this.so -> reference/lib/libtuscany_sca_reference.so
> > > xsd/
> > > ws-binding.xsd
> > > reference/
> > > lib/
> > > libtuscany_sca_reference.so
> > > service/
> > > lib/
> > > libtuscany_sca_service.so
> > >
> > > Here the runtime loads anything in any folder named 'extension' so the
> > > reference extension would be loaded/initialized.
> >
> > Windows doesn't really do symlinks - I guess on windows we'd have the
> > tuscany_sca_reference.dll in extension/ and nothing in reference/lib/
> > or reference/bin/? For extensions that have dependencies on other
> > libraries, those dll's could go in lib/
>
>
> I would actually have a copy of the dll in extension on windows and delete
> it to disable. For a large extension it would be very wasy to make a tiny
> dll which just implements the initialise interface expected by the runtime
> and leave everything else in a separate dll in bin/lib.
>
> > I would package the cpp language extension as
> > >
> > > cpp/
> > > bin/
> > > lib/
> > > include/
> > > xsd/
> > > extension/
> > >
> > > The bin, lib, include are exactly what you would expect from a package
> > that
> > > you might want to build against. The extension/ folder would hold the
> > > library that implements the extension interface and xsd holds... well
> > xsds!
> > >
> >
> > All sounds good to me. +1
> >
> > Andy
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >
> >
>
>
> --
> Pete
>
> Ok, so I think I get it. My concern is that giving the developer the
flexibility to put "extension" and "xsd" anywhere they like means that they
will accidentally duplicate these directories as these are pretty common
names for directories. So we could ask them always to have them at the top
level. Doesn't remove the possibility for duplication but reduces the risk a
little. Or we could make the names a little more exotic with some
qualification "sca_extension", "sca_xsd" for example, and maintain the
flexibility you want.
Simon
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Andrew Borley <aj...@gmail.com> wrote:
>
> On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > >
> > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > >
> > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > > >
> > > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > >
> > > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 30/11/06, Jean-Sebastien Delfino < jsdelfino@apache.org>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Pete Robbins wrote:
> > > > > > > > > > Our current method of packaging and loading an extension
> is
> > > > > > fairly
> > > > > > > > > > simple:
> > > > > > > > > > we load all schema and libraries in the extensions path.
> > > This
> > > > > > has a
> > > > > > > > > > number
> > > > > > > > > > of problems.
> > > > > > > > > >
> > > > > > > > > > 1. An extension may consist of more than one library e.g
> .
> > > > > > > > > > libmy_extension.so
> > > > > > > > > > and libmy_extension_utils.so. Our current loading scheme
> > > will
> > > > > > > attempt
> > > > > > > > > to
> > > > > > > > > > load both of these and may fail on the one that doesn't
> > > > provide
> > > > > > the
> > > > > > > > > > extension initialize method. On MacOS the output of our
> > > build
> > > > > > > produces
> > > > > > > > > a
> > > > > > > > > > libx.dylib and a series of symlinks to this called
> > > > libx.0.dylib,
> > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which
> > > > doesn't
> > > > > > > cause
> > > > > > > > > > problems as they are all the sam library and just
> repeatedly
> > > > > > > register
> > > > > > > > > to
> > > > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > > > >
> > > > > > > > > > 2. Control of whether or not to load an extension
> library is
> > > > > > > currently
> > > > > > > > > by
> > > > > > > > > > renaming the library so the runtime doesn't find it. An
> > > > example
> > > > > > is
> > > > > > > > > > that we
> > > > > > > > > > ship our python extension as
> > > libtuscany_sca_python.so.diabled.
> > > > > > This
> > > > > > > is
> > > > > > > > >
> > > > > > > > > > horrible and error prone.
> > > > > > > > > >
> > > > > > > > > > We could improve this by having a system configuration
> file
> > > > that
> > > > > > > lists
> > > > > > > > > > the
> > > > > > > > > > required extensions but the I like the self contained
> > > package
> > > > > > > approach
> > > > > > > > > > that
> > > > > > > > > > we have now. I'd like to implement an improved scheme
> for
> > > > > > packaging
> > > > > > > an
> > > > > > > > > > extension by introducing a per extension configuration
> file.
> > > > > > > Something
> > > > > > > > > > like:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > > > </scacppp:extension>
> > > > > > > > > >
> > > > > > > > > > So the package would look like:
> > > > > > > > > >
> > > > > > > > > > extensions/
> > > > > > > > > > ws/
> > > > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > > > lib/
> > > > > > > > > > xsd/
> > > > > > > > > > other_folder/
> > > > > > > > > > ...
> > > > > > > > > >
> > > > > > > > > > The .extension configuration file is saying to load the
> > > > library
> > > > > > > which
> > > > > > > > > is
> > > > > > > > > > located somewhere in the package... the runtime will
> find
> > > > it...
> > > > > > no
> > > > > > > > > > need to
> > > > > > > > > > specify a path.
> > > > > > > > > >
> > > > > > > > > > Taking this further the configuration file could list
> the
> > > > schema
> > > > > > to
> > > > > > > be
> > > > > > > > > > loaded. Currently the runtime will just load any it
> finds
> > > but
> > > > > > these
> > > > > > > > > > may not
> > > > > > > > > > be needed by the runtime e.g. the schema may be for some
> > > > > > extension
> > > > > > > > > > implementation specific purpose.
> > > > > > > > > >
> > > > > > > > > > I think it would also be good for the extension
> > > > initialization()
> > > > > > > > > > method to
> > > > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > > > extension("/tuscany/extensions/ws"). This would allow
> the
> > > > > > extension
> > > > > > > > > > package
> > > > > > > > > > to contain any configuration information that it needs.
> > > > > > > > > >
> > > > > > > > > > I'd like to start by at least introducing the .extension
> > > file
> > > > > > for
> > > > > > > each
> > > > > > > > > > extension and loading only the specified library(ies) if
> the
> > > > > > > extension
> > > > > > > > > is
> > > > > > > > > > enabled.
> > > > > > > > > >
> > > > > > > > > > Any thoughts?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Two thoughts:
> > > > > > > > > - convention over configuration
> > > > > > > > > - the runtime should be consumable without having to go
> tweak
> > > > XML
> > > > > > > > > configuration files
> > > > > > > > >
> > > > > > > > > If I remember correctly, renaming the dlls to .disabled
> was a
> > > > last
> > > > > >
> > > > > > > > > minute change to work around DLL loading errors with our
> M2
> > > > > > release on
> > > > > > > > > Windows.
> > > > > > > > >
> > > > > > > > > I agree that we should do better than renaming to
> .disabled,
> > > but
> > > > > > I'd
> > > > > > > > > like to understand better the actual issues that we faced
> > > before
> > > > > > > > > inventing yet another XML based runtime configuration
> > > language
> > > > :)
> > > > > > > > >
> > > > > > > > > I'm aware of the following issues:
> > > > > > > > > 1. We need the runtime to load extension libs only, not
> other
> > > > libs
> > > > > > > under
> > > > > > > > >
> > > > > > > > > the extension directory which are not actual extensions
> > > > > > > > > 2. Same for XSDs, we need to load XSDs that contribute to
> > > SCDL,
> > > > > > and
> > > > > > > > > leave other XSDs under the extension directory alone
> > > > > > > > > 3. Extensions that cannot be loaded because some of their
> > > > > > dependencies
> > > > > > > > > are not there should no break the runtime
> > > > > > > > > 4. The system admin / installer should be able to disable
> > > > > > extensions
> > > > > > > > > that won't load because their dependencies are not there
> (I'm
> > > > not
> > > > > > yet
> > > > > > > > > convinced that this is still an issue if we manage to
> solve
> > > > issue
> > > > > > #3)
> > > > > > > > >
> > > > > > > > > Did we run into any other big issues?
> > > > > > > >
> > > > > > > >
> > > > > > > > No, that's about it.
> > > > > > > >
> > > > > > > > 2. XSD loading can be done by convention (schemas for the
> > > runtime
> > > > > > are
> > > > > > > > always in a folder called 'xsd'
> > > > > > > > 1. Could be solved in a similar way by only loading
> libraries in
> > > a
> > > > > > > folder
> > > > > > > > called ???
> > > > > > > > 3. Can be solved by just ignoring the load errors/issuing a
> > > > warning
> > > > > > > rather
> > > > > > > > than giving up
> > > > > > > > 4. Can be solved by solution to 3.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > I recall now why 3. was a big problem. Windows sometimes
> throws up
> > > > an
> > > > > > > error
> > > > > > > dialog when the load fails so it was not just a case of the
> > > runtime
> > > > > > > handling
> > > > > > > the load failure so we had to "disable" the extension.
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > --
> > > > > > > Pete
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Pete
> > > > > > >
> > > > > > > Pete, just summarizing so that I understand. It seems there
> are
> > > two
> > > > > > parts
> > > > > > to this.
> > > > > >
> > > > > > A/ Building and installing an extension in order that it can be
> > > > consumed
> > > > > > by
> > > > > > the runtime. Building and installing extensions is potentially a
> > > > > > separate
> > > > > > activity from building and installing the runtime itself.
> Looking at
> > > > the
> > > > > > M2
> > > > > > download they all come together at the moment.
> > > > >
> > > > >
> > > > > Yes. It just happens that we are the only providers of extensions
> at
> > > the
> > > > > moment.
> > > > >
> > > > > B/ Optionally consuming/enabling an extension in the runtime, once
> the
> > > > > > extension has been installed, in such a way that the runtime is
> able
> > > > to
> > > > > > find
> > > > > > and load it.
> > > > >
> > > > >
> > > > > Yes.
> > > > >
> > > > > If I understand the previous posts (and the current code) A/ is
> > > achieved
> > > > > > by
> > > > > > building the extension in the deployed folder layout
> > > > > > extension type/
> > > > > > bin
> > > > > > lib
> > > > > > xsd
> > > > >
> > > > >
> > > > > The current runtime does not mandate any layout. It will seek and
> > > > > destroy.. I mean seek and load any library it finds and any .xsd
> it
> > > > finds.
> > > > > That layout is just how we package extensions in ou build at the
> > > moment.
> > > > >
> > > > >
> > > > > and Pete you are proposing that this is adjusted so that there are
> > > > > > directories that hold just the stuff that the contains the
> extension
> > > > > > rather
> > > > > > than the things it depends on.
> > > > > > extension type/
> > > > > > bin
> > > > > > extensionbin?
> > > > > > lib
> > > > > > xsd
> > > > > > extensionxsd?
> > > > > >
> > > > > > You are also proposing that B/ is achieved by ensuring that this
> > > > > > directory
> > > > > > structure be placed in a location that the the runtime can
> search
> > > for
> > > > > > active
> > > > > > extensions (as it does at the moment in the deploy directory)
> > > > >
> > > > >
> > > > > it currently loads any extensions from under the
> > > > > <sca_install_dir>/extensions folder
> > > > >
> > > > > and that the
> > > > > > runtime ignores any badly configured extensions if possible.
> > > Removing
> > > > an
> > > > > > extension's directory from the deploy
> > > > >
> > > > >
> > > > > extensions not deploy
> > > > >
> > > > > directory structure has the effect of
> > > > > > disabling it so I guess this is the fallback if the runtime
> can't
> > > > > > continue.
> > > > >
> > > > >
> > > > > That is true in today's implementation.
> > > > >
> > > > > Anything more complex and, as Jean-Sebastien suggests, you start
> > > getting
> > > > > > into the full blown package and dependency management problem
> that
> > > > many
> > > > > > other systems try to solve in different ways, e.g. rpm or pear.
> > > > >
> > > > >
> > > > > Agreed... I'll give up on that plan ;-)
> > > > >
> > > > >
> > > > > What we need is to enable the runtime to identify which extension
> > > > > libraries to load and which schema to load. My suggestion is:
> > > > >
> > > > > my_extension/
> > > > > bin/
> > > > > extension/
> > > > > library_that_will_be_loaded_by_the_runtime.so
> > > > > lib/
> > > > > lib_my_extension.so
> > > > > include/
> > > > > ... some headers maybe
> > > > > xsd/
> > > > > my_binding.xsd
> > > > > any_other_folder/
> > > > > any_stuff_I_like.xsd
> > > > >
> > > > > So the runtime will load any xsd that is in a folder named xsd and
> > > > attempt
> > > > > to load any library in a folder called extension. It would not
> attempt
> > > > to
> > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > --
> > > > > Pete
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Pete
> > > >
> > > > Sounds like the right idea to me. What goes in the bin dir?.
> >
> >
> >
> > Anything you like!
>
>
> On windows, .exe's and .dll's are often found in bin, whereas the
> .lib's (used for linking against at compile time) are generally found
> in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which
> is more unix-like, and, IMO, nicer)
>
> >
> > Should the
> > > lib be...
> > >
> > > lib/
> > > lib_my_extension.lib
> >
> >
> > lif you like... again... anything you like.
> >
> > Maybe you could stick with bin,include, lib,xsd and then have your
> "other"
> > > directory have everything else in it that the extension might rely on.
> > > Depends what you now anticipate being in bin and lib over and above
> the
> > > actual library exposing the extension.
> > >
> > > Will you continue to mandate a specific place where extensions are
> > > installed, i.e. from you post <sca_install_dir>/extensions folder?
> >
> >
> > Yes
> >
> > So that
> > > you can move an extension out of there if you don't want it to load.
> >
> >
> > Well the idea here is you don't need to remove the whole package..just
> the
> > library from the extension dir
> >
> > Simon
> > >
> > >
> > The layout of the extension package is whatever the package creater
> wants to
> > make it. We don't care. All we will care about is loading any library in
> a
> > folder anywhere in the extension package that is in a folder called
> > 'extension' and expecting it to implement the extension initialize
> method.
> > We will also load any schema files in any folder called 'xsd'. An
> extension
> > only needs the xsd folder if it has schema required for the runtime
> model.
> >
> > As an example here is the ws binding extension implemented for axis2c:
> >
> > <sca_install>/extensions/
> > ... somewhere in this tree... could be anywhere...
> > ws/
> > xsd/
> > ws-binding.xsd
> > reference/
> > lib/
> > libtuscany_sca_reference.so
> > service/
> > lib/
> > libtuscany_sca_service.so
> >
> >
> > OK... today the runtime would load the ws-binding.xsd as it loads any
> xsd it
> > finds. It would also load any .so it finds so both reference aand
> service
> > extensions would be loaded/initialized.
> >
> > With my proposed change the xsd would be loaded (as it's in a folder
> named
> > 'xsd' ) but the libraries would not. To get the extensions loaded you
> would
> > have to package with an 'extension' folder ... again anywhere in the
> tree...
> > which "contains" the libraries to load eg.
> >
> > ws/
> > extension/
> > libload_this.so -> reference/lib/libtuscany_sca_reference.so
> > xsd/
> > ws-binding.xsd
> > reference/
> > lib/
> > libtuscany_sca_reference.so
> > service/
> > lib/
> > libtuscany_sca_service.so
> >
> > Here the runtime loads anything in any folder named 'extension' so the
> > reference extension would be loaded/initialized.
>
> Windows doesn't really do symlinks - I guess on windows we'd have the
> tuscany_sca_reference.dll in extension/ and nothing in reference/lib/
> or reference/bin/? For extensions that have dependencies on other
> libraries, those dll's could go in lib/
I would actually have a copy of the dll in extension on windows and delete
it to disable. For a large extension it would be very wasy to make a tiny
dll which just implements the initialise interface expected by the runtime
and leave everything else in a separate dll in bin/lib.
> I would package the cpp language extension as
> >
> > cpp/
> > bin/
> > lib/
> > include/
> > xsd/
> > extension/
> >
> > The bin, lib, include are exactly what you would expect from a package
> that
> > you might want to build against. The extension/ folder would hold the
> > library that implements the extension interface and xsd holds... well
> xsds!
> >
>
> All sounds good to me. +1
>
> Andy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Andrew Borley <aj...@gmail.com>.
On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> >
> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >
> > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > >
> > > >
> > > >
> > > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > > >
> > > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > >
> > > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 30/11/06, Jean-Sebastien Delfino < jsdelfino@apache.org>
> > > wrote:
> > > > > > > >
> > > > > > > > Pete Robbins wrote:
> > > > > > > > > Our current method of packaging and loading an extension is
> > > > > fairly
> > > > > > > > > simple:
> > > > > > > > > we load all schema and libraries in the extensions path.
> > This
> > > > > has a
> > > > > > > > > number
> > > > > > > > > of problems.
> > > > > > > > >
> > > > > > > > > 1. An extension may consist of more than one library e.g.
> > > > > > > > > libmy_extension.so
> > > > > > > > > and libmy_extension_utils.so. Our current loading scheme
> > will
> > > > > > attempt
> > > > > > > > to
> > > > > > > > > load both of these and may fail on the one that doesn't
> > > provide
> > > > > the
> > > > > > > > > extension initialize method. On MacOS the output of our
> > build
> > > > > > produces
> > > > > > > > a
> > > > > > > > > libx.dylib and a series of symlinks to this called
> > > libx.0.dylib,
> > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which
> > > doesn't
> > > > > > cause
> > > > > > > > > problems as they are all the sam library and just repeatedly
> > > > > > register
> > > > > > > > to
> > > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > > >
> > > > > > > > > 2. Control of whether or not to load an extension library is
> > > > > > currently
> > > > > > > > by
> > > > > > > > > renaming the library so the runtime doesn't find it. An
> > > example
> > > > > is
> > > > > > > > > that we
> > > > > > > > > ship our python extension as
> > libtuscany_sca_python.so.diabled.
> > > > > This
> > > > > > is
> > > > > > > >
> > > > > > > > > horrible and error prone.
> > > > > > > > >
> > > > > > > > > We could improve this by having a system configuration file
> > > that
> > > > > > lists
> > > > > > > > > the
> > > > > > > > > required extensions but the I like the self contained
> > package
> > > > > > approach
> > > > > > > > > that
> > > > > > > > > we have now. I'd like to implement an improved scheme for
> > > > > packaging
> > > > > > an
> > > > > > > > > extension by introducing a per extension configuration file.
> > > > > > Something
> > > > > > > > > like:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > > </scacppp:extension>
> > > > > > > > >
> > > > > > > > > So the package would look like:
> > > > > > > > >
> > > > > > > > > extensions/
> > > > > > > > > ws/
> > > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > > lib/
> > > > > > > > > xsd/
> > > > > > > > > other_folder/
> > > > > > > > > ...
> > > > > > > > >
> > > > > > > > > The .extension configuration file is saying to load the
> > > library
> > > > > > which
> > > > > > > > is
> > > > > > > > > located somewhere in the package... the runtime will find
> > > it...
> > > > > no
> > > > > > > > > need to
> > > > > > > > > specify a path.
> > > > > > > > >
> > > > > > > > > Taking this further the configuration file could list the
> > > schema
> > > > > to
> > > > > > be
> > > > > > > > > loaded. Currently the runtime will just load any it finds
> > but
> > > > > these
> > > > > > > > > may not
> > > > > > > > > be needed by the runtime e.g. the schema may be for some
> > > > > extension
> > > > > > > > > implementation specific purpose.
> > > > > > > > >
> > > > > > > > > I think it would also be good for the extension
> > > initialization()
> > > > > > > > > method to
> > > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > > extension("/tuscany/extensions/ws"). This would allow the
> > > > > extension
> > > > > > > > > package
> > > > > > > > > to contain any configuration information that it needs.
> > > > > > > > >
> > > > > > > > > I'd like to start by at least introducing the .extension
> > file
> > > > > for
> > > > > > each
> > > > > > > > > extension and loading only the specified library(ies) if the
> > > > > > extension
> > > > > > > > is
> > > > > > > > > enabled.
> > > > > > > > >
> > > > > > > > > Any thoughts?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > Two thoughts:
> > > > > > > > - convention over configuration
> > > > > > > > - the runtime should be consumable without having to go tweak
> > > XML
> > > > > > > > configuration files
> > > > > > > >
> > > > > > > > If I remember correctly, renaming the dlls to .disabled was a
> > > last
> > > > >
> > > > > > > > minute change to work around DLL loading errors with our M2
> > > > > release on
> > > > > > > > Windows.
> > > > > > > >
> > > > > > > > I agree that we should do better than renaming to .disabled,
> > but
> > > > > I'd
> > > > > > > > like to understand better the actual issues that we faced
> > before
> > > > > > > > inventing yet another XML based runtime configuration
> > language
> > > :)
> > > > > > > >
> > > > > > > > I'm aware of the following issues:
> > > > > > > > 1. We need the runtime to load extension libs only, not other
> > > libs
> > > > > > under
> > > > > > > >
> > > > > > > > the extension directory which are not actual extensions
> > > > > > > > 2. Same for XSDs, we need to load XSDs that contribute to
> > SCDL,
> > > > > and
> > > > > > > > leave other XSDs under the extension directory alone
> > > > > > > > 3. Extensions that cannot be loaded because some of their
> > > > > dependencies
> > > > > > > > are not there should no break the runtime
> > > > > > > > 4. The system admin / installer should be able to disable
> > > > > extensions
> > > > > > > > that won't load because their dependencies are not there (I'm
> > > not
> > > > > yet
> > > > > > > > convinced that this is still an issue if we manage to solve
> > > issue
> > > > > #3)
> > > > > > > >
> > > > > > > > Did we run into any other big issues?
> > > > > > >
> > > > > > >
> > > > > > > No, that's about it.
> > > > > > >
> > > > > > > 2. XSD loading can be done by convention (schemas for the
> > runtime
> > > > > are
> > > > > > > always in a folder called 'xsd'
> > > > > > > 1. Could be solved in a similar way by only loading libraries in
> > a
> > > > > > folder
> > > > > > > called ???
> > > > > > > 3. Can be solved by just ignoring the load errors/issuing a
> > > warning
> > > > > > rather
> > > > > > > than giving up
> > > > > > > 4. Can be solved by solution to 3.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > I recall now why 3. was a big problem. Windows sometimes throws up
> > > an
> > > > > > error
> > > > > > dialog when the load fails so it was not just a case of the
> > runtime
> > > > > > handling
> > > > > > the load failure so we had to "disable" the extension.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > --
> > > > > > Pete
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Pete
> > > > > >
> > > > > > Pete, just summarizing so that I understand. It seems there are
> > two
> > > > > parts
> > > > > to this.
> > > > >
> > > > > A/ Building and installing an extension in order that it can be
> > > consumed
> > > > > by
> > > > > the runtime. Building and installing extensions is potentially a
> > > > > separate
> > > > > activity from building and installing the runtime itself. Looking at
> > > the
> > > > > M2
> > > > > download they all come together at the moment.
> > > >
> > > >
> > > > Yes. It just happens that we are the only providers of extensions at
> > the
> > > > moment.
> > > >
> > > > B/ Optionally consuming/enabling an extension in the runtime, once the
> > > > > extension has been installed, in such a way that the runtime is able
> > > to
> > > > > find
> > > > > and load it.
> > > >
> > > >
> > > > Yes.
> > > >
> > > > If I understand the previous posts (and the current code) A/ is
> > achieved
> > > > > by
> > > > > building the extension in the deployed folder layout
> > > > > extension type/
> > > > > bin
> > > > > lib
> > > > > xsd
> > > >
> > > >
> > > > The current runtime does not mandate any layout. It will seek and
> > > > destroy.. I mean seek and load any library it finds and any .xsd it
> > > finds.
> > > > That layout is just how we package extensions in ou build at the
> > moment.
> > > >
> > > >
> > > > and Pete you are proposing that this is adjusted so that there are
> > > > > directories that hold just the stuff that the contains the extension
> > > > > rather
> > > > > than the things it depends on.
> > > > > extension type/
> > > > > bin
> > > > > extensionbin?
> > > > > lib
> > > > > xsd
> > > > > extensionxsd?
> > > > >
> > > > > You are also proposing that B/ is achieved by ensuring that this
> > > > > directory
> > > > > structure be placed in a location that the the runtime can search
> > for
> > > > > active
> > > > > extensions (as it does at the moment in the deploy directory)
> > > >
> > > >
> > > > it currently loads any extensions from under the
> > > > <sca_install_dir>/extensions folder
> > > >
> > > > and that the
> > > > > runtime ignores any badly configured extensions if possible.
> > Removing
> > > an
> > > > > extension's directory from the deploy
> > > >
> > > >
> > > > extensions not deploy
> > > >
> > > > directory structure has the effect of
> > > > > disabling it so I guess this is the fallback if the runtime can't
> > > > > continue.
> > > >
> > > >
> > > > That is true in today's implementation.
> > > >
> > > > Anything more complex and, as Jean-Sebastien suggests, you start
> > getting
> > > > > into the full blown package and dependency management problem that
> > > many
> > > > > other systems try to solve in different ways, e.g. rpm or pear.
> > > >
> > > >
> > > > Agreed... I'll give up on that plan ;-)
> > > >
> > > >
> > > > What we need is to enable the runtime to identify which extension
> > > > libraries to load and which schema to load. My suggestion is:
> > > >
> > > > my_extension/
> > > > bin/
> > > > extension/
> > > > library_that_will_be_loaded_by_the_runtime.so
> > > > lib/
> > > > lib_my_extension.so
> > > > include/
> > > > ... some headers maybe
> > > > xsd/
> > > > my_binding.xsd
> > > > any_other_folder/
> > > > any_stuff_I_like.xsd
> > > >
> > > > So the runtime will load any xsd that is in a folder named xsd and
> > > attempt
> > > > to load any library in a folder called extension. It would not attempt
> > > to
> > > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > > >
> > > >
> > > > Cheers,
> > > >
> > > > --
> > > > Pete
> > > >
> > >
> > >
> > >
> > > --
> > > Pete
> > >
> > > Sounds like the right idea to me. What goes in the bin dir?.
>
>
>
> Anything you like!
On windows, .exe's and .dll's are often found in bin, whereas the
.lib's (used for linking against at compile time) are generally found
in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which
is more unix-like, and, IMO, nicer)
>
> Should the
> > lib be...
> >
> > lib/
> > lib_my_extension.lib
>
>
> lif you like... again... anything you like.
>
> Maybe you could stick with bin,include, lib,xsd and then have your "other"
> > directory have everything else in it that the extension might rely on.
> > Depends what you now anticipate being in bin and lib over and above the
> > actual library exposing the extension.
> >
> > Will you continue to mandate a specific place where extensions are
> > installed, i.e. from you post <sca_install_dir>/extensions folder?
>
>
> Yes
>
> So that
> > you can move an extension out of there if you don't want it to load.
>
>
> Well the idea here is you don't need to remove the whole package..just the
> library from the extension dir
>
> Simon
> >
> >
> The layout of the extension package is whatever the package creater wants to
> make it. We don't care. All we will care about is loading any library in a
> folder anywhere in the extension package that is in a folder called
> 'extension' and expecting it to implement the extension initialize method.
> We will also load any schema files in any folder called 'xsd'. An extension
> only needs the xsd folder if it has schema required for the runtime model.
>
> As an example here is the ws binding extension implemented for axis2c:
>
> <sca_install>/extensions/
> ... somewhere in this tree... could be anywhere...
> ws/
> xsd/
> ws-binding.xsd
> reference/
> lib/
> libtuscany_sca_reference.so
> service/
> lib/
> libtuscany_sca_service.so
>
>
> OK... today the runtime would load the ws-binding.xsd as it loads any xsd it
> finds. It would also load any .so it finds so both reference aand service
> extensions would be loaded/initialized.
>
> With my proposed change the xsd would be loaded (as it's in a folder named
> 'xsd' ) but the libraries would not. To get the extensions loaded you would
> have to package with an 'extension' folder ... again anywhere in the tree...
> which "contains" the libraries to load eg.
>
> ws/
> extension/
> libload_this.so -> reference/lib/libtuscany_sca_reference.so
> xsd/
> ws-binding.xsd
> reference/
> lib/
> libtuscany_sca_reference.so
> service/
> lib/
> libtuscany_sca_service.so
>
> Here the runtime loads anything in any folder named 'extension' so the
> reference extension would be loaded/initialized.
Windows doesn't really do symlinks - I guess on windows we'd have the
tuscany_sca_reference.dll in extension/ and nothing in reference/lib/
or reference/bin/? For extensions that have dependencies on other
libraries, those dll's could go in lib/
> I would package the cpp language extension as
>
> cpp/
> bin/
> lib/
> include/
> xsd/
> extension/
>
> The bin, lib, include are exactly what you would expect from a package that
> you might want to build against. The extension/ folder would hold the
> library that implements the extension interface and xsd holds... well xsds!
>
All sounds good to me. +1
Andy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
>
> On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >
> > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >
> > >
> > >
> > > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > > >
> > > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > >
> > > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 30/11/06, Jean-Sebastien Delfino < jsdelfino@apache.org>
> > wrote:
> > > > > > >
> > > > > > > Pete Robbins wrote:
> > > > > > > > Our current method of packaging and loading an extension is
> > > > fairly
> > > > > > > > simple:
> > > > > > > > we load all schema and libraries in the extensions path.
> This
> > > > has a
> > > > > > > > number
> > > > > > > > of problems.
> > > > > > > >
> > > > > > > > 1. An extension may consist of more than one library e.g.
> > > > > > > > libmy_extension.so
> > > > > > > > and libmy_extension_utils.so. Our current loading scheme
> will
> > > > > attempt
> > > > > > > to
> > > > > > > > load both of these and may fail on the one that doesn't
> > provide
> > > > the
> > > > > > > > extension initialize method. On MacOS the output of our
> build
> > > > > produces
> > > > > > > a
> > > > > > > > libx.dylib and a series of symlinks to this called
> > libx.0.dylib,
> > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which
> > doesn't
> > > > > cause
> > > > > > > > problems as they are all the sam library and just repeatedly
> > > > > register
> > > > > > > to
> > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > >
> > > > > > > > 2. Control of whether or not to load an extension library is
> > > > > currently
> > > > > > > by
> > > > > > > > renaming the library so the runtime doesn't find it. An
> > example
> > > > is
> > > > > > > > that we
> > > > > > > > ship our python extension as
> libtuscany_sca_python.so.diabled.
> > > > This
> > > > > is
> > > > > > >
> > > > > > > > horrible and error prone.
> > > > > > > >
> > > > > > > > We could improve this by having a system configuration file
> > that
> > > > > lists
> > > > > > > > the
> > > > > > > > required extensions but the I like the self contained
> package
> > > > > approach
> > > > > > > > that
> > > > > > > > we have now. I'd like to implement an improved scheme for
> > > > packaging
> > > > > an
> > > > > > > > extension by introducing a per extension configuration file.
> > > > > Something
> > > > > > > > like:
> > > > > > > >
> > > > > > > >
> > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > >
> > > > > > > >
> > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > </scacppp:extension>
> > > > > > > >
> > > > > > > > So the package would look like:
> > > > > > > >
> > > > > > > > extensions/
> > > > > > > > ws/
> > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > lib/
> > > > > > > > xsd/
> > > > > > > > other_folder/
> > > > > > > > ...
> > > > > > > >
> > > > > > > > The .extension configuration file is saying to load the
> > library
> > > > > which
> > > > > > > is
> > > > > > > > located somewhere in the package... the runtime will find
> > it...
> > > > no
> > > > > > > > need to
> > > > > > > > specify a path.
> > > > > > > >
> > > > > > > > Taking this further the configuration file could list the
> > schema
> > > > to
> > > > > be
> > > > > > > > loaded. Currently the runtime will just load any it finds
> but
> > > > these
> > > > > > > > may not
> > > > > > > > be needed by the runtime e.g. the schema may be for some
> > > > extension
> > > > > > > > implementation specific purpose.
> > > > > > > >
> > > > > > > > I think it would also be good for the extension
> > initialization()
> > > > > > > > method to
> > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > extension("/tuscany/extensions/ws"). This would allow the
> > > > extension
> > > > > > > > package
> > > > > > > > to contain any configuration information that it needs.
> > > > > > > >
> > > > > > > > I'd like to start by at least introducing the .extension
> file
> > > > for
> > > > > each
> > > > > > > > extension and loading only the specified library(ies) if the
> > > > > extension
> > > > > > > is
> > > > > > > > enabled.
> > > > > > > >
> > > > > > > > Any thoughts?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > Two thoughts:
> > > > > > > - convention over configuration
> > > > > > > - the runtime should be consumable without having to go tweak
> > XML
> > > > > > > configuration files
> > > > > > >
> > > > > > > If I remember correctly, renaming the dlls to .disabled was a
> > last
> > > >
> > > > > > > minute change to work around DLL loading errors with our M2
> > > > release on
> > > > > > > Windows.
> > > > > > >
> > > > > > > I agree that we should do better than renaming to .disabled,
> but
> > > > I'd
> > > > > > > like to understand better the actual issues that we faced
> before
> > > > > > > inventing yet another XML based runtime configuration
> language
> > :)
> > > > > > >
> > > > > > > I'm aware of the following issues:
> > > > > > > 1. We need the runtime to load extension libs only, not other
> > libs
> > > > > under
> > > > > > >
> > > > > > > the extension directory which are not actual extensions
> > > > > > > 2. Same for XSDs, we need to load XSDs that contribute to
> SCDL,
> > > > and
> > > > > > > leave other XSDs under the extension directory alone
> > > > > > > 3. Extensions that cannot be loaded because some of their
> > > > dependencies
> > > > > > > are not there should no break the runtime
> > > > > > > 4. The system admin / installer should be able to disable
> > > > extensions
> > > > > > > that won't load because their dependencies are not there (I'm
> > not
> > > > yet
> > > > > > > convinced that this is still an issue if we manage to solve
> > issue
> > > > #3)
> > > > > > >
> > > > > > > Did we run into any other big issues?
> > > > > >
> > > > > >
> > > > > > No, that's about it.
> > > > > >
> > > > > > 2. XSD loading can be done by convention (schemas for the
> runtime
> > > > are
> > > > > > always in a folder called 'xsd'
> > > > > > 1. Could be solved in a similar way by only loading libraries in
> a
> > > > > folder
> > > > > > called ???
> > > > > > 3. Can be solved by just ignoring the load errors/issuing a
> > warning
> > > > > rather
> > > > > > than giving up
> > > > > > 4. Can be solved by solution to 3.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > I recall now why 3. was a big problem. Windows sometimes throws up
> > an
> > > > > error
> > > > > dialog when the load fails so it was not just a case of the
> runtime
> > > > > handling
> > > > > the load failure so we had to "disable" the extension.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > --
> > > > > Pete
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pete
> > > > >
> > > > > Pete, just summarizing so that I understand. It seems there are
> two
> > > > parts
> > > > to this.
> > > >
> > > > A/ Building and installing an extension in order that it can be
> > consumed
> > > > by
> > > > the runtime. Building and installing extensions is potentially a
> > > > separate
> > > > activity from building and installing the runtime itself. Looking at
> > the
> > > > M2
> > > > download they all come together at the moment.
> > >
> > >
> > > Yes. It just happens that we are the only providers of extensions at
> the
> > > moment.
> > >
> > > B/ Optionally consuming/enabling an extension in the runtime, once the
> > > > extension has been installed, in such a way that the runtime is able
> > to
> > > > find
> > > > and load it.
> > >
> > >
> > > Yes.
> > >
> > > If I understand the previous posts (and the current code) A/ is
> achieved
> > > > by
> > > > building the extension in the deployed folder layout
> > > > extension type/
> > > > bin
> > > > lib
> > > > xsd
> > >
> > >
> > > The current runtime does not mandate any layout. It will seek and
> > > destroy.. I mean seek and load any library it finds and any .xsd it
> > finds.
> > > That layout is just how we package extensions in ou build at the
> moment.
> > >
> > >
> > > and Pete you are proposing that this is adjusted so that there are
> > > > directories that hold just the stuff that the contains the extension
> > > > rather
> > > > than the things it depends on.
> > > > extension type/
> > > > bin
> > > > extensionbin?
> > > > lib
> > > > xsd
> > > > extensionxsd?
> > > >
> > > > You are also proposing that B/ is achieved by ensuring that this
> > > > directory
> > > > structure be placed in a location that the the runtime can search
> for
> > > > active
> > > > extensions (as it does at the moment in the deploy directory)
> > >
> > >
> > > it currently loads any extensions from under the
> > > <sca_install_dir>/extensions folder
> > >
> > > and that the
> > > > runtime ignores any badly configured extensions if possible.
> Removing
> > an
> > > > extension's directory from the deploy
> > >
> > >
> > > extensions not deploy
> > >
> > > directory structure has the effect of
> > > > disabling it so I guess this is the fallback if the runtime can't
> > > > continue.
> > >
> > >
> > > That is true in today's implementation.
> > >
> > > Anything more complex and, as Jean-Sebastien suggests, you start
> getting
> > > > into the full blown package and dependency management problem that
> > many
> > > > other systems try to solve in different ways, e.g. rpm or pear.
> > >
> > >
> > > Agreed... I'll give up on that plan ;-)
> > >
> > >
> > > What we need is to enable the runtime to identify which extension
> > > libraries to load and which schema to load. My suggestion is:
> > >
> > > my_extension/
> > > bin/
> > > extension/
> > > library_that_will_be_loaded_by_the_runtime.so
> > > lib/
> > > lib_my_extension.so
> > > include/
> > > ... some headers maybe
> > > xsd/
> > > my_binding.xsd
> > > any_other_folder/
> > > any_stuff_I_like.xsd
> > >
> > > So the runtime will load any xsd that is in a folder named xsd and
> > attempt
> > > to load any library in a folder called extension. It would not attempt
> > to
> > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > >
> > >
> > > Cheers,
> > >
> > > --
> > > Pete
> > >
> >
> >
> >
> > --
> > Pete
> >
> > Sounds like the right idea to me. What goes in the bin dir?.
Anything you like!
Should the
> lib be...
>
> lib/
> lib_my_extension.lib
lif you like... again... anything you like.
Maybe you could stick with bin,include, lib,xsd and then have your "other"
> directory have everything else in it that the extension might rely on.
> Depends what you now anticipate being in bin and lib over and above the
> actual library exposing the extension.
>
> Will you continue to mandate a specific place where extensions are
> installed, i.e. from you post <sca_install_dir>/extensions folder?
Yes
So that
> you can move an extension out of there if you don't want it to load.
Well the idea here is you don't need to remove the whole package..just the
library from the extension dir
Simon
>
>
The layout of the extension package is whatever the package creater wants to
make it. We don't care. All we will care about is loading any library in a
folder anywhere in the extension package that is in a folder called
'extension' and expecting it to implement the extension initialize method.
We will also load any schema files in any folder called 'xsd'. An extension
only needs the xsd folder if it has schema required for the runtime model.
As an example here is the ws binding extension implemented for axis2c:
<sca_install>/extensions/
... somewhere in this tree... could be anywhere...
ws/
xsd/
ws-binding.xsd
reference/
lib/
libtuscany_sca_reference.so
service/
lib/
libtuscany_sca_service.so
OK... today the runtime would load the ws-binding.xsd as it loads any xsd it
finds. It would also load any .so it finds so both reference aand service
extensions would be loaded/initialized.
With my proposed change the xsd would be loaded (as it's in a folder named
'xsd' ) but the libraries would not. To get the extensions loaded you would
have to package with an 'extension' folder ... again anywhere in the tree...
which "contains" the libraries to load eg.
ws/
extension/
libload_this.so -> reference/lib/libtuscany_sca_reference.so
xsd/
ws-binding.xsd
reference/
lib/
libtuscany_sca_reference.so
service/
lib/
libtuscany_sca_service.so
Here the runtime loads anything in any folder named 'extension' so the
reference extension would be loaded/initialized.
I would package the cpp language extension as
cpp/
bin/
lib/
include/
xsd/
extension/
The bin, lib, include are exactly what you would expect from a package that
you might want to build against. The extension/ folder would hold the
library that implements the extension interface and xsd holds... well xsds!
Cheers,
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Simon Laws <si...@googlemail.com>.
On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>
> On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> >
> >
> >
> > On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> > >
> > > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > >
> > > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > On 30/11/06, Jean-Sebastien Delfino < jsdelfino@apache.org>
> wrote:
> > > > > >
> > > > > > Pete Robbins wrote:
> > > > > > > Our current method of packaging and loading an extension is
> > > fairly
> > > > > > > simple:
> > > > > > > we load all schema and libraries in the extensions path. This
> > > has a
> > > > > > > number
> > > > > > > of problems.
> > > > > > >
> > > > > > > 1. An extension may consist of more than one library e.g.
> > > > > > > libmy_extension.so
> > > > > > > and libmy_extension_utils.so. Our current loading scheme will
> > > > attempt
> > > > > > to
> > > > > > > load both of these and may fail on the one that doesn't
> provide
> > > the
> > > > > > > extension initialize method. On MacOS the output of our build
> > > > produces
> > > > > > a
> > > > > > > libx.dylib and a series of symlinks to this called
> libx.0.dylib,
> > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which
> doesn't
> > > > cause
> > > > > > > problems as they are all the sam library and just repeatedly
> > > > register
> > > > > > to
> > > > > > > handle the same requests. Very inefficient though!
> > > > > > >
> > > > > > > 2. Control of whether or not to load an extension library is
> > > > currently
> > > > > > by
> > > > > > > renaming the library so the runtime doesn't find it. An
> example
> > > is
> > > > > > > that we
> > > > > > > ship our python extension as libtuscany_sca_python.so.diabled.
> > > This
> > > > is
> > > > > >
> > > > > > > horrible and error prone.
> > > > > > >
> > > > > > > We could improve this by having a system configuration file
> that
> > > > lists
> > > > > > > the
> > > > > > > required extensions but the I like the self contained package
> > > > approach
> > > > > > > that
> > > > > > > we have now. I'd like to implement an improved scheme for
> > > packaging
> > > > an
> > > > > > > extension by introducing a per extension configuration file.
> > > > Something
> > > > > > > like:
> > > > > > >
> > > > > > >
> > > > > > > tuscany_sca_ws_binding.extension
> > > > > > >
> > > > > > >
> > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > </scacppp:extension>
> > > > > > >
> > > > > > > So the package would look like:
> > > > > > >
> > > > > > > extensions/
> > > > > > > ws/
> > > > > > > tusany_sca_ws_binding.extension
> > > > > > > lib/
> > > > > > > xsd/
> > > > > > > other_folder/
> > > > > > > ...
> > > > > > >
> > > > > > > The .extension configuration file is saying to load the
> library
> > > > which
> > > > > > is
> > > > > > > located somewhere in the package... the runtime will find
> it...
> > > no
> > > > > > > need to
> > > > > > > specify a path.
> > > > > > >
> > > > > > > Taking this further the configuration file could list the
> schema
> > > to
> > > > be
> > > > > > > loaded. Currently the runtime will just load any it finds but
> > > these
> > > > > > > may not
> > > > > > > be needed by the runtime e.g. the schema may be for some
> > > extension
> > > > > > > implementation specific purpose.
> > > > > > >
> > > > > > > I think it would also be good for the extension
> initialization()
> > > > > > > method to
> > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > extension("/tuscany/extensions/ws"). This would allow the
> > > extension
> > > > > > > package
> > > > > > > to contain any configuration information that it needs.
> > > > > > >
> > > > > > > I'd like to start by at least introducing the .extension file
> > > for
> > > > each
> > > > > > > extension and loading only the specified library(ies) if the
> > > > extension
> > > > > > is
> > > > > > > enabled.
> > > > > > >
> > > > > > > Any thoughts?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > Two thoughts:
> > > > > > - convention over configuration
> > > > > > - the runtime should be consumable without having to go tweak
> XML
> > > > > > configuration files
> > > > > >
> > > > > > If I remember correctly, renaming the dlls to .disabled was a
> last
> > >
> > > > > > minute change to work around DLL loading errors with our M2
> > > release on
> > > > > > Windows.
> > > > > >
> > > > > > I agree that we should do better than renaming to .disabled, but
> > > I'd
> > > > > > like to understand better the actual issues that we faced before
> > > > > > inventing yet another XML based runtime configuration language
> :)
> > > > > >
> > > > > > I'm aware of the following issues:
> > > > > > 1. We need the runtime to load extension libs only, not other
> libs
> > > > under
> > > > > >
> > > > > > the extension directory which are not actual extensions
> > > > > > 2. Same for XSDs, we need to load XSDs that contribute to SCDL,
> > > and
> > > > > > leave other XSDs under the extension directory alone
> > > > > > 3. Extensions that cannot be loaded because some of their
> > > dependencies
> > > > > > are not there should no break the runtime
> > > > > > 4. The system admin / installer should be able to disable
> > > extensions
> > > > > > that won't load because their dependencies are not there (I'm
> not
> > > yet
> > > > > > convinced that this is still an issue if we manage to solve
> issue
> > > #3)
> > > > > >
> > > > > > Did we run into any other big issues?
> > > > >
> > > > >
> > > > > No, that's about it.
> > > > >
> > > > > 2. XSD loading can be done by convention (schemas for the runtime
> > > are
> > > > > always in a folder called 'xsd'
> > > > > 1. Could be solved in a similar way by only loading libraries in a
> > > > folder
> > > > > called ???
> > > > > 3. Can be solved by just ignoring the load errors/issuing a
> warning
> > > > rather
> > > > > than giving up
> > > > > 4. Can be solved by solution to 3.
> > > > >
> > > > >
> > > > >
> > > >
> > > > I recall now why 3. was a big problem. Windows sometimes throws up
> an
> > > > error
> > > > dialog when the load fails so it was not just a case of the runtime
> > > > handling
> > > > the load failure so we had to "disable" the extension.
> > > >
> > > > Cheers,
> > > >
> > > > --
> > > > Pete
> > > >
> > > >
> > > >
> > > > --
> > > > Pete
> > > >
> > > > Pete, just summarizing so that I understand. It seems there are two
> > > parts
> > > to this.
> > >
> > > A/ Building and installing an extension in order that it can be
> consumed
> > > by
> > > the runtime. Building and installing extensions is potentially a
> > > separate
> > > activity from building and installing the runtime itself. Looking at
> the
> > > M2
> > > download they all come together at the moment.
> >
> >
> > Yes. It just happens that we are the only providers of extensions at the
> > moment.
> >
> > B/ Optionally consuming/enabling an extension in the runtime, once the
> > > extension has been installed, in such a way that the runtime is able
> to
> > > find
> > > and load it.
> >
> >
> > Yes.
> >
> > If I understand the previous posts (and the current code) A/ is achieved
> > > by
> > > building the extension in the deployed folder layout
> > > extension type/
> > > bin
> > > lib
> > > xsd
> >
> >
> > The current runtime does not mandate any layout. It will seek and
> > destroy.. I mean seek and load any library it finds and any .xsd it
> finds.
> > That layout is just how we package extensions in ou build at the moment.
> >
> >
> > and Pete you are proposing that this is adjusted so that there are
> > > directories that hold just the stuff that the contains the extension
> > > rather
> > > than the things it depends on.
> > > extension type/
> > > bin
> > > extensionbin?
> > > lib
> > > xsd
> > > extensionxsd?
> > >
> > > You are also proposing that B/ is achieved by ensuring that this
> > > directory
> > > structure be placed in a location that the the runtime can search for
> > > active
> > > extensions (as it does at the moment in the deploy directory)
> >
> >
> > it currently loads any extensions from under the
> > <sca_install_dir>/extensions folder
> >
> > and that the
> > > runtime ignores any badly configured extensions if possible. Removing
> an
> > > extension's directory from the deploy
> >
> >
> > extensions not deploy
> >
> > directory structure has the effect of
> > > disabling it so I guess this is the fallback if the runtime can't
> > > continue.
> >
> >
> > That is true in today's implementation.
> >
> > Anything more complex and, as Jean-Sebastien suggests, you start getting
> > > into the full blown package and dependency management problem that
> many
> > > other systems try to solve in different ways, e.g. rpm or pear.
> >
> >
> > Agreed... I'll give up on that plan ;-)
> >
> >
> > What we need is to enable the runtime to identify which extension
> > libraries to load and which schema to load. My suggestion is:
> >
> > my_extension/
> > bin/
> > extension/
> > library_that_will_be_loaded_by_the_runtime.so
> > lib/
> > lib_my_extension.so
> > include/
> > ... some headers maybe
> > xsd/
> > my_binding.xsd
> > any_other_folder/
> > any_stuff_I_like.xsd
> >
> > So the runtime will load any xsd that is in a folder named xsd and
> attempt
> > to load any library in a folder called extension. It would not attempt
> to
> > load the any_stuff_I_like.xsd or the lib_my_extension.so
> >
> >
> > Cheers,
> >
> > --
> > Pete
> >
>
>
>
> --
> Pete
>
> Sounds like the right idea to me. What goes in the bin dir?. Should the
lib be...
lib/
lib_my_extension.lib
Maybe you could stick with bin,include, lib,xsd and then have your "other"
directory have everything else in it that the extension might rely on.
Depends what you now anticipate being in bin and lib over and above the
actual library exposing the extension.
Will you continue to mandate a specific place where extensions are
installed, i.e. from you post <sca_install_dir>/extensions folder? So that
you can move an extension out of there if you don't want it to load.
Simon
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
>
>
>
> On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
> >
> > On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >
> > > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > > >
> > > >
> > > >
> > > > On 30/11/06, Jean-Sebastien Delfino < jsdelfino@apache.org> wrote:
> > > > >
> > > > > Pete Robbins wrote:
> > > > > > Our current method of packaging and loading an extension is
> > fairly
> > > > > > simple:
> > > > > > we load all schema and libraries in the extensions path. This
> > has a
> > > > > > number
> > > > > > of problems.
> > > > > >
> > > > > > 1. An extension may consist of more than one library e.g.
> > > > > > libmy_extension.so
> > > > > > and libmy_extension_utils.so. Our current loading scheme will
> > > attempt
> > > > > to
> > > > > > load both of these and may fail on the one that doesn't provide
> > the
> > > > > > extension initialize method. On MacOS the output of our build
> > > produces
> > > > > a
> > > > > > libx.dylib and a series of symlinks to this called libx.0.dylib,
> > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't
> > > cause
> > > > > > problems as they are all the sam library and just repeatedly
> > > register
> > > > > to
> > > > > > handle the same requests. Very inefficient though!
> > > > > >
> > > > > > 2. Control of whether or not to load an extension library is
> > > currently
> > > > > by
> > > > > > renaming the library so the runtime doesn't find it. An example
> > is
> > > > > > that we
> > > > > > ship our python extension as libtuscany_sca_python.so.diabled.
> > This
> > > is
> > > > >
> > > > > > horrible and error prone.
> > > > > >
> > > > > > We could improve this by having a system configuration file that
> > > lists
> > > > > > the
> > > > > > required extensions but the I like the self contained package
> > > approach
> > > > > > that
> > > > > > we have now. I'd like to implement an improved scheme for
> > packaging
> > > an
> > > > > > extension by introducing a per extension configuration file.
> > > Something
> > > > > > like:
> > > > > >
> > > > > >
> > > > > > tuscany_sca_ws_binding.extension
> > > > > >
> > > > > >
> > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > </scacppp:extension>
> > > > > >
> > > > > > So the package would look like:
> > > > > >
> > > > > > extensions/
> > > > > > ws/
> > > > > > tusany_sca_ws_binding.extension
> > > > > > lib/
> > > > > > xsd/
> > > > > > other_folder/
> > > > > > ...
> > > > > >
> > > > > > The .extension configuration file is saying to load the library
> > > which
> > > > > is
> > > > > > located somewhere in the package... the runtime will find it...
> > no
> > > > > > need to
> > > > > > specify a path.
> > > > > >
> > > > > > Taking this further the configuration file could list the schema
> > to
> > > be
> > > > > > loaded. Currently the runtime will just load any it finds but
> > these
> > > > > > may not
> > > > > > be needed by the runtime e.g. the schema may be for some
> > extension
> > > > > > implementation specific purpose.
> > > > > >
> > > > > > I think it would also be good for the extension initialization()
> > > > > > method to
> > > > > > take as a parameter the root of the extension e.g.
> > > > > > extension("/tuscany/extensions/ws"). This would allow the
> > extension
> > > > > > package
> > > > > > to contain any configuration information that it needs.
> > > > > >
> > > > > > I'd like to start by at least introducing the .extension file
> > for
> > > each
> > > > > > extension and loading only the specified library(ies) if the
> > > extension
> > > > > is
> > > > > > enabled.
> > > > > >
> > > > > > Any thoughts?
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > Two thoughts:
> > > > > - convention over configuration
> > > > > - the runtime should be consumable without having to go tweak XML
> > > > > configuration files
> > > > >
> > > > > If I remember correctly, renaming the dlls to .disabled was a last
> >
> > > > > minute change to work around DLL loading errors with our M2
> > release on
> > > > > Windows.
> > > > >
> > > > > I agree that we should do better than renaming to .disabled, but
> > I'd
> > > > > like to understand better the actual issues that we faced before
> > > > > inventing yet another XML based runtime configuration language :)
> > > > >
> > > > > I'm aware of the following issues:
> > > > > 1. We need the runtime to load extension libs only, not other libs
> > > under
> > > > >
> > > > > the extension directory which are not actual extensions
> > > > > 2. Same for XSDs, we need to load XSDs that contribute to SCDL,
> > and
> > > > > leave other XSDs under the extension directory alone
> > > > > 3. Extensions that cannot be loaded because some of their
> > dependencies
> > > > > are not there should no break the runtime
> > > > > 4. The system admin / installer should be able to disable
> > extensions
> > > > > that won't load because their dependencies are not there (I'm not
> > yet
> > > > > convinced that this is still an issue if we manage to solve issue
> > #3)
> > > > >
> > > > > Did we run into any other big issues?
> > > >
> > > >
> > > > No, that's about it.
> > > >
> > > > 2. XSD loading can be done by convention (schemas for the runtime
> > are
> > > > always in a folder called 'xsd'
> > > > 1. Could be solved in a similar way by only loading libraries in a
> > > folder
> > > > called ???
> > > > 3. Can be solved by just ignoring the load errors/issuing a warning
> > > rather
> > > > than giving up
> > > > 4. Can be solved by solution to 3.
> > > >
> > > >
> > > >
> > >
> > > I recall now why 3. was a big problem. Windows sometimes throws up an
> > > error
> > > dialog when the load fails so it was not just a case of the runtime
> > > handling
> > > the load failure so we had to "disable" the extension.
> > >
> > > Cheers,
> > >
> > > --
> > > Pete
> > >
> > >
> > >
> > > --
> > > Pete
> > >
> > > Pete, just summarizing so that I understand. It seems there are two
> > parts
> > to this.
> >
> > A/ Building and installing an extension in order that it can be consumed
> > by
> > the runtime. Building and installing extensions is potentially a
> > separate
> > activity from building and installing the runtime itself. Looking at the
> > M2
> > download they all come together at the moment.
>
>
> Yes. It just happens that we are the only providers of extensions at the
> moment.
>
> B/ Optionally consuming/enabling an extension in the runtime, once the
> > extension has been installed, in such a way that the runtime is able to
> > find
> > and load it.
>
>
> Yes.
>
> If I understand the previous posts (and the current code) A/ is achieved
> > by
> > building the extension in the deployed folder layout
> > extension type/
> > bin
> > lib
> > xsd
>
>
> The current runtime does not mandate any layout. It will seek and
> destroy.. I mean seek and load any library it finds and any .xsd it finds.
> That layout is just how we package extensions in ou build at the moment.
>
>
> and Pete you are proposing that this is adjusted so that there are
> > directories that hold just the stuff that the contains the extension
> > rather
> > than the things it depends on.
> > extension type/
> > bin
> > extensionbin?
> > lib
> > xsd
> > extensionxsd?
> >
> > You are also proposing that B/ is achieved by ensuring that this
> > directory
> > structure be placed in a location that the the runtime can search for
> > active
> > extensions (as it does at the moment in the deploy directory)
>
>
> it currently loads any extensions from under the
> <sca_install_dir>/extensions folder
>
> and that the
> > runtime ignores any badly configured extensions if possible. Removing an
> > extension's directory from the deploy
>
>
> extensions not deploy
>
> directory structure has the effect of
> > disabling it so I guess this is the fallback if the runtime can't
> > continue.
>
>
> That is true in today's implementation.
>
> Anything more complex and, as Jean-Sebastien suggests, you start getting
> > into the full blown package and dependency management problem that many
> > other systems try to solve in different ways, e.g. rpm or pear.
>
>
> Agreed... I'll give up on that plan ;-)
>
>
> What we need is to enable the runtime to identify which extension
> libraries to load and which schema to load. My suggestion is:
>
> my_extension/
> bin/
> extension/
> library_that_will_be_loaded_by_the_runtime.so
> lib/
> lib_my_extension.so
> include/
> ... some headers maybe
> xsd/
> my_binding.xsd
> any_other_folder/
> any_stuff_I_like.xsd
>
> So the runtime will load any xsd that is in a folder named xsd and attempt
> to load any library in a folder called extension. It would not attempt to
> load the any_stuff_I_like.xsd or the lib_my_extension.so
>
>
> Cheers,
>
> --
> Pete
>
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Simon Laws <si...@googlemail.com> wrote:
>
> On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
> >
> > On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> > >
> > >
> > >
> > > On 30/11/06, Jean-Sebastien Delfino <js...@apache.org> wrote:
> > > >
> > > > Pete Robbins wrote:
> > > > > Our current method of packaging and loading an extension is fairly
> > > > > simple:
> > > > > we load all schema and libraries in the extensions path. This has
> a
> > > > > number
> > > > > of problems.
> > > > >
> > > > > 1. An extension may consist of more than one library e.g.
> > > > > libmy_extension.so
> > > > > and libmy_extension_utils.so. Our current loading scheme will
> > attempt
> > > > to
> > > > > load both of these and may fail on the one that doesn't provide
> the
> > > > > extension initialize method. On MacOS the output of our build
> > produces
> > > > a
> > > > > libx.dylib and a series of symlinks to this called libx.0.dylib ,
> > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't
> > cause
> > > > > problems as they are all the sam library and just repeatedly
> > register
> > > > to
> > > > > handle the same requests. Very inefficient though!
> > > > >
> > > > > 2. Control of whether or not to load an extension library is
> > currently
> > > > by
> > > > > renaming the library so the runtime doesn't find it. An example is
> > > > > that we
> > > > > ship our python extension as libtuscany_sca_python.so.diabled.
> This
> > is
> > > >
> > > > > horrible and error prone.
> > > > >
> > > > > We could improve this by having a system configuration file that
> > lists
> > > > > the
> > > > > required extensions but the I like the self contained package
> > approach
> > > > > that
> > > > > we have now. I'd like to implement an improved scheme for
> packaging
> > an
> > > > > extension by introducing a per extension configuration file.
> > Something
> > > > > like:
> > > > >
> > > > >
> > > > > tuscany_sca_ws_binding.extension
> > > > >
> > > > >
> > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > <library name="tuscany_sca_ws_service"/>
> > > > > </scacppp:extension>
> > > > >
> > > > > So the package would look like:
> > > > >
> > > > > extensions/
> > > > > ws/
> > > > > tusany_sca_ws_binding.extension
> > > > > lib/
> > > > > xsd/
> > > > > other_folder/
> > > > > ...
> > > > >
> > > > > The .extension configuration file is saying to load the library
> > which
> > > > is
> > > > > located somewhere in the package... the runtime will find it... no
> > > > > need to
> > > > > specify a path.
> > > > >
> > > > > Taking this further the configuration file could list the schema
> to
> > be
> > > > > loaded. Currently the runtime will just load any it finds but
> these
> > > > > may not
> > > > > be needed by the runtime e.g. the schema may be for some extension
> > > > > implementation specific purpose.
> > > > >
> > > > > I think it would also be good for the extension initialization()
> > > > > method to
> > > > > take as a parameter the root of the extension e.g.
> > > > > extension("/tuscany/extensions/ws"). This would allow the
> extension
> > > > > package
> > > > > to contain any configuration information that it needs.
> > > > >
> > > > > I'd like to start by at least introducing the .extension file for
> > each
> > > > > extension and loading only the specified library(ies) if the
> > extension
> > > > is
> > > > > enabled.
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > >
> > > > >
> > > >
> > > > Two thoughts:
> > > > - convention over configuration
> > > > - the runtime should be consumable without having to go tweak XML
> > > > configuration files
> > > >
> > > > If I remember correctly, renaming the dlls to .disabled was a last
> > > > minute change to work around DLL loading errors with our M2 release
> on
> > > > Windows.
> > > >
> > > > I agree that we should do better than renaming to .disabled, but I'd
> > > > like to understand better the actual issues that we faced before
> > > > inventing yet another XML based runtime configuration language :)
> > > >
> > > > I'm aware of the following issues:
> > > > 1. We need the runtime to load extension libs only, not other libs
> > under
> > > >
> > > > the extension directory which are not actual extensions
> > > > 2. Same for XSDs, we need to load XSDs that contribute to SCDL, and
> > > > leave other XSDs under the extension directory alone
> > > > 3. Extensions that cannot be loaded because some of their
> dependencies
> > > > are not there should no break the runtime
> > > > 4. The system admin / installer should be able to disable extensions
> > > > that won't load because their dependencies are not there (I'm not
> yet
> > > > convinced that this is still an issue if we manage to solve issue
> #3)
> > > >
> > > > Did we run into any other big issues?
> > >
> > >
> > > No, that's about it.
> > >
> > > 2. XSD loading can be done by convention (schemas for the runtime are
> > > always in a folder called 'xsd'
> > > 1. Could be solved in a similar way by only loading libraries in a
> > folder
> > > called ???
> > > 3. Can be solved by just ignoring the load errors/issuing a warning
> > rather
> > > than giving up
> > > 4. Can be solved by solution to 3.
> > >
> > >
> > >
> >
> > I recall now why 3. was a big problem. Windows sometimes throws up an
> > error
> > dialog when the load fails so it was not just a case of the runtime
> > handling
> > the load failure so we had to "disable" the extension.
> >
> > Cheers,
> >
> > --
> > Pete
> >
> >
> >
> > --
> > Pete
> >
> > Pete, just summarizing so that I understand. It seems there are two
> parts
> to this.
>
> A/ Building and installing an extension in order that it can be consumed
> by
> the runtime. Building and installing extensions is potentially a separate
> activity from building and installing the runtime itself. Looking at the
> M2
> download they all come together at the moment.
Yes. It just happens that we are the only providers of extensions at the
moment.
B/ Optionally consuming/enabling an extension in the runtime, once the
> extension has been installed, in such a way that the runtime is able to
> find
> and load it.
Yes.
If I understand the previous posts (and the current code) A/ is achieved by
> building the extension in the deployed folder layout
> extension type/
> bin
> lib
> xsd
The current runtime does not mandate any layout. It will seek and destroy..
I mean seek and load any library it finds and any .xsd it finds. That layout
is just how we package extensions in ou build at the moment.
and Pete you are proposing that this is adjusted so that there are
> directories that hold just the stuff that the contains the extension
> rather
> than the things it depends on.
> extension type/
> bin
> extensionbin?
> lib
> xsd
> extensionxsd?
>
> You are also proposing that B/ is achieved by ensuring that this directory
> structure be placed in a location that the the runtime can search for
> active
> extensions (as it does at the moment in the deploy directory)
it currently loads any extensions from under the
<sca_install_dir>/extensions folder
and that the
> runtime ignores any badly configured extensions if possible. Removing an
> extension's directory from the deploy
extensions not deploy
directory structure has the effect of
> disabling it so I guess this is the fallback if the runtime can't
> continue.
That is true in today's implementation.
Anything more complex and, as Jean-Sebastien suggests, you start getting
> into the full blown package and dependency management problem that many
> other systems try to solve in different ways, e.g. rpm or pear.
Agreed... I'll give up on that plan ;-)
What we need is to enable the runtime to identify which extension libraries
to load and which schema to load. My suggestion is:
my_extension/
bin/
extension/
library_that_will_be_loaded_by_the_runtime.so
lib/
lib_my_extension.so
include/
... some headers maybe
xsd/
my_binding.xsd
any_other_folder/
any_stuff_I_like.xsd
So the runtime will load any xsd that is in a folder named xsd and attempt
to load any library in a folder called extension. It would not attempt to
load the any_stuff_I_like.xsd or the lib_my_extension.so
Cheers,
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Simon Laws <si...@googlemail.com>.
On 11/30/06, Pete Robbins <ro...@googlemail.com> wrote:
>
> On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
> >
> >
> >
> > On 30/11/06, Jean-Sebastien Delfino <js...@apache.org> wrote:
> > >
> > > Pete Robbins wrote:
> > > > Our current method of packaging and loading an extension is fairly
> > > > simple:
> > > > we load all schema and libraries in the extensions path. This has a
> > > > number
> > > > of problems.
> > > >
> > > > 1. An extension may consist of more than one library e.g.
> > > > libmy_extension.so
> > > > and libmy_extension_utils.so. Our current loading scheme will
> attempt
> > > to
> > > > load both of these and may fail on the one that doesn't provide the
> > > > extension initialize method. On MacOS the output of our build
> produces
> > > a
> > > > libx.dylib and a series of symlinks to this called libx.0.dylib ,
> > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't
> cause
> > > > problems as they are all the sam library and just repeatedly
> register
> > > to
> > > > handle the same requests. Very inefficient though!
> > > >
> > > > 2. Control of whether or not to load an extension library is
> currently
> > > by
> > > > renaming the library so the runtime doesn't find it. An example is
> > > > that we
> > > > ship our python extension as libtuscany_sca_python.so.diabled. This
> is
> > >
> > > > horrible and error prone.
> > > >
> > > > We could improve this by having a system configuration file that
> lists
> > > > the
> > > > required extensions but the I like the self contained package
> approach
> > > > that
> > > > we have now. I'd like to implement an improved scheme for packaging
> an
> > > > extension by introducing a per extension configuration file.
> Something
> > > > like:
> > > >
> > > >
> > > > tuscany_sca_ws_binding.extension
> > > >
> > > >
> > > > <scacpp:extension name="ws binding" enabled="true">
> > > > <library name="tuscany_sca_ws_reference"/>
> > > > <library name="tuscany_sca_ws_service"/>
> > > > </scacppp:extension>
> > > >
> > > > So the package would look like:
> > > >
> > > > extensions/
> > > > ws/
> > > > tusany_sca_ws_binding.extension
> > > > lib/
> > > > xsd/
> > > > other_folder/
> > > > ...
> > > >
> > > > The .extension configuration file is saying to load the library
> which
> > > is
> > > > located somewhere in the package... the runtime will find it... no
> > > > need to
> > > > specify a path.
> > > >
> > > > Taking this further the configuration file could list the schema to
> be
> > > > loaded. Currently the runtime will just load any it finds but these
> > > > may not
> > > > be needed by the runtime e.g. the schema may be for some extension
> > > > implementation specific purpose.
> > > >
> > > > I think it would also be good for the extension initialization()
> > > > method to
> > > > take as a parameter the root of the extension e.g.
> > > > extension("/tuscany/extensions/ws"). This would allow the extension
> > > > package
> > > > to contain any configuration information that it needs.
> > > >
> > > > I'd like to start by at least introducing the .extension file for
> each
> > > > extension and loading only the specified library(ies) if the
> extension
> > > is
> > > > enabled.
> > > >
> > > > Any thoughts?
> > > >
> > > >
> > > >
> > >
> > > Two thoughts:
> > > - convention over configuration
> > > - the runtime should be consumable without having to go tweak XML
> > > configuration files
> > >
> > > If I remember correctly, renaming the dlls to .disabled was a last
> > > minute change to work around DLL loading errors with our M2 release on
> > > Windows.
> > >
> > > I agree that we should do better than renaming to .disabled, but I'd
> > > like to understand better the actual issues that we faced before
> > > inventing yet another XML based runtime configuration language :)
> > >
> > > I'm aware of the following issues:
> > > 1. We need the runtime to load extension libs only, not other libs
> under
> > >
> > > the extension directory which are not actual extensions
> > > 2. Same for XSDs, we need to load XSDs that contribute to SCDL, and
> > > leave other XSDs under the extension directory alone
> > > 3. Extensions that cannot be loaded because some of their dependencies
> > > are not there should no break the runtime
> > > 4. The system admin / installer should be able to disable extensions
> > > that won't load because their dependencies are not there (I'm not yet
> > > convinced that this is still an issue if we manage to solve issue #3)
> > >
> > > Did we run into any other big issues?
> >
> >
> > No, that's about it.
> >
> > 2. XSD loading can be done by convention (schemas for the runtime are
> > always in a folder called 'xsd'
> > 1. Could be solved in a similar way by only loading libraries in a
> folder
> > called ???
> > 3. Can be solved by just ignoring the load errors/issuing a warning
> rather
> > than giving up
> > 4. Can be solved by solution to 3.
> >
> >
> >
>
> I recall now why 3. was a big problem. Windows sometimes throws up an
> error
> dialog when the load fails so it was not just a case of the runtime
> handling
> the load failure so we had to "disable" the extension.
>
> Cheers,
>
> --
> Pete
>
>
>
> --
> Pete
>
> Pete, just summarizing so that I understand. It seems there are two parts
to this.
A/ Building and installing an extension in order that it can be consumed by
the runtime. Building and installing extensions is potentially a separate
activity from building and installing the runtime itself. Looking at the M2
download they all come together at the moment.
B/ Optionally consuming/enabling an extension in the runtime, once the
extension has been installed, in such a way that the runtime is able to find
and load it.
If I understand the previous posts (and the current code) A/ is achieved by
building the extension in the deployed folder layout
extension type/
bin
lib
xsd
and Pete you are proposing that this is adjusted so that there are
directories that hold just the stuff that the contains the extension rather
than the things it depends on.
extension type/
bin
extensionbin?
lib
xsd
extensionxsd?
You are also proposing that B/ is achieved by ensuring that this directory
structure be placed in a location that the the runtime can search for active
extensions (as it does at the moment in the deploy directory) and that the
runtime ignores any badly configured extensions if possible. Removing an
extension's directory from the deploy directory structure has the effect of
disabling it so I guess this is the fallback if the runtime can't continue.
Anything more complex and, as Jean-Sebastien suggests, you start getting
into the full blown package and dependency management problem that many
other systems try to solve in different ways, e.g. rpm or pear.
Simon
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Pete Robbins <ro...@googlemail.com> wrote:
>
>
>
> On 30/11/06, Jean-Sebastien Delfino <js...@apache.org> wrote:
> >
> > Pete Robbins wrote:
> > > Our current method of packaging and loading an extension is fairly
> > > simple:
> > > we load all schema and libraries in the extensions path. This has a
> > > number
> > > of problems.
> > >
> > > 1. An extension may consist of more than one library e.g.
> > > libmy_extension.so
> > > and libmy_extension_utils.so. Our current loading scheme will attempt
> > to
> > > load both of these and may fail on the one that doesn't provide the
> > > extension initialize method. On MacOS the output of our build produces
> > a
> > > libx.dylib and a series of symlinks to this called libx.0.dylib ,
> > > libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't cause
> > > problems as they are all the sam library and just repeatedly register
> > to
> > > handle the same requests. Very inefficient though!
> > >
> > > 2. Control of whether or not to load an extension library is currently
> > by
> > > renaming the library so the runtime doesn't find it. An example is
> > > that we
> > > ship our python extension as libtuscany_sca_python.so.diabled. This is
> >
> > > horrible and error prone.
> > >
> > > We could improve this by having a system configuration file that lists
> > > the
> > > required extensions but the I like the self contained package approach
> > > that
> > > we have now. I'd like to implement an improved scheme for packaging an
> > > extension by introducing a per extension configuration file. Something
> > > like:
> > >
> > >
> > > tuscany_sca_ws_binding.extension
> > >
> > >
> > > <scacpp:extension name="ws binding" enabled="true">
> > > <library name="tuscany_sca_ws_reference"/>
> > > <library name="tuscany_sca_ws_service"/>
> > > </scacppp:extension>
> > >
> > > So the package would look like:
> > >
> > > extensions/
> > > ws/
> > > tusany_sca_ws_binding.extension
> > > lib/
> > > xsd/
> > > other_folder/
> > > ...
> > >
> > > The .extension configuration file is saying to load the library which
> > is
> > > located somewhere in the package... the runtime will find it... no
> > > need to
> > > specify a path.
> > >
> > > Taking this further the configuration file could list the schema to be
> > > loaded. Currently the runtime will just load any it finds but these
> > > may not
> > > be needed by the runtime e.g. the schema may be for some extension
> > > implementation specific purpose.
> > >
> > > I think it would also be good for the extension initialization()
> > > method to
> > > take as a parameter the root of the extension e.g.
> > > extension("/tuscany/extensions/ws"). This would allow the extension
> > > package
> > > to contain any configuration information that it needs.
> > >
> > > I'd like to start by at least introducing the .extension file for each
> > > extension and loading only the specified library(ies) if the extension
> > is
> > > enabled.
> > >
> > > Any thoughts?
> > >
> > >
> > >
> >
> > Two thoughts:
> > - convention over configuration
> > - the runtime should be consumable without having to go tweak XML
> > configuration files
> >
> > If I remember correctly, renaming the dlls to .disabled was a last
> > minute change to work around DLL loading errors with our M2 release on
> > Windows.
> >
> > I agree that we should do better than renaming to .disabled, but I'd
> > like to understand better the actual issues that we faced before
> > inventing yet another XML based runtime configuration language :)
> >
> > I'm aware of the following issues:
> > 1. We need the runtime to load extension libs only, not other libs under
> >
> > the extension directory which are not actual extensions
> > 2. Same for XSDs, we need to load XSDs that contribute to SCDL, and
> > leave other XSDs under the extension directory alone
> > 3. Extensions that cannot be loaded because some of their dependencies
> > are not there should no break the runtime
> > 4. The system admin / installer should be able to disable extensions
> > that won't load because their dependencies are not there (I'm not yet
> > convinced that this is still an issue if we manage to solve issue #3)
> >
> > Did we run into any other big issues?
>
>
> No, that's about it.
>
> 2. XSD loading can be done by convention (schemas for the runtime are
> always in a folder called 'xsd'
> 1. Could be solved in a similar way by only loading libraries in a folder
> called ???
> 3. Can be solved by just ignoring the load errors/issuing a warning rather
> than giving up
> 4. Can be solved by solution to 3.
>
>
>
I recall now why 3. was a big problem. Windows sometimes throws up an error
dialog when the load fails so it was not just a case of the runtime handling
the load failure so we had to "disable" the extension.
Cheers,
--
Pete
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Pete Robbins <ro...@googlemail.com>.
On 30/11/06, Jean-Sebastien Delfino <js...@apache.org> wrote:
>
> Pete Robbins wrote:
> > Our current method of packaging and loading an extension is fairly
> > simple:
> > we load all schema and libraries in the extensions path. This has a
> > number
> > of problems.
> >
> > 1. An extension may consist of more than one library e.g.
> > libmy_extension.so
> > and libmy_extension_utils.so. Our current loading scheme will attempt to
> > load both of these and may fail on the one that doesn't provide the
> > extension initialize method. On MacOS the output of our build produces a
> > libx.dylib and a series of symlinks to this called libx.0.dylib,
> > libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't cause
> > problems as they are all the sam library and just repeatedly register to
> > handle the same requests. Very inefficient though!
> >
> > 2. Control of whether or not to load an extension library is currently
> by
> > renaming the library so the runtime doesn't find it. An example is
> > that we
> > ship our python extension as libtuscany_sca_python.so.diabled. This is
> > horrible and error prone.
> >
> > We could improve this by having a system configuration file that lists
> > the
> > required extensions but the I like the self contained package approach
> > that
> > we have now. I'd like to implement an improved scheme for packaging an
> > extension by introducing a per extension configuration file. Something
> > like:
> >
> >
> > tuscany_sca_ws_binding.extension
> >
> >
> > <scacpp:extension name="ws binding" enabled="true">
> > <library name="tuscany_sca_ws_reference"/>
> > <library name="tuscany_sca_ws_service"/>
> > </scacppp:extension>
> >
> > So the package would look like:
> >
> > extensions/
> > ws/
> > tusany_sca_ws_binding.extension
> > lib/
> > xsd/
> > other_folder/
> > ...
> >
> > The .extension configuration file is saying to load the library which is
> > located somewhere in the package... the runtime will find it... no
> > need to
> > specify a path.
> >
> > Taking this further the configuration file could list the schema to be
> > loaded. Currently the runtime will just load any it finds but these
> > may not
> > be needed by the runtime e.g. the schema may be for some extension
> > implementation specific purpose.
> >
> > I think it would also be good for the extension initialization()
> > method to
> > take as a parameter the root of the extension e.g.
> > extension("/tuscany/extensions/ws"). This would allow the extension
> > package
> > to contain any configuration information that it needs.
> >
> > I'd like to start by at least introducing the .extension file for each
> > extension and loading only the specified library(ies) if the extension
> is
> > enabled.
> >
> > Any thoughts?
> >
> >
> >
>
> Two thoughts:
> - convention over configuration
> - the runtime should be consumable without having to go tweak XML
> configuration files
>
> If I remember correctly, renaming the dlls to .disabled was a last
> minute change to work around DLL loading errors with our M2 release on
> Windows.
>
> I agree that we should do better than renaming to .disabled, but I'd
> like to understand better the actual issues that we faced before
> inventing yet another XML based runtime configuration language :)
>
> I'm aware of the following issues:
> 1. We need the runtime to load extension libs only, not other libs under
> the extension directory which are not actual extensions
> 2. Same for XSDs, we need to load XSDs that contribute to SCDL, and
> leave other XSDs under the extension directory alone
> 3. Extensions that cannot be loaded because some of their dependencies
> are not there should no break the runtime
> 4. The system admin / installer should be able to disable extensions
> that won't load because their dependencies are not there (I'm not yet
> convinced that this is still an issue if we manage to solve issue #3)
>
> Did we run into any other big issues?
No, that's about it.
2. XSD loading can be done by convention (schemas for the runtime are always
in a folder called 'xsd'
1. Could be solved in a similar way by only loading libraries in a folder
called ???
3. Can be solved by just ignoring the load errors/issuing a warning rather
than giving up
4. Can be solved by solution to 3.
--
Pete
Re: [C++] Binding/Language Extensions - packaging
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Pete Robbins wrote:
> Our current method of packaging and loading an extension is fairly
> simple:
> we load all schema and libraries in the extensions path. This has a
> number
> of problems.
>
> 1. An extension may consist of more than one library e.g.
> libmy_extension.so
> and libmy_extension_utils.so. Our current loading scheme will attempt to
> load both of these and may fail on the one that doesn't provide the
> extension initialize method. On MacOS the output of our build produces a
> libx.dylib and a series of symlinks to this called libx.0.dylib,
> libx.0.0.dylib etc.. ur runtime loads ALL of these which doesn't cause
> problems as they are all the sam library and just repeatedly register to
> handle the same requests. Very inefficient though!
>
> 2. Control of whether or not to load an extension library is currently by
> renaming the library so the runtime doesn't find it. An example is
> that we
> ship our python extension as libtuscany_sca_python.so.diabled. This is
> horrible and error prone.
>
> We could improve this by having a system configuration file that lists
> the
> required extensions but the I like the self contained package approach
> that
> we have now. I'd like to implement an improved scheme for packaging an
> extension by introducing a per extension configuration file. Something
> like:
>
>
> tuscany_sca_ws_binding.extension
>
>
> <scacpp:extension name="ws binding" enabled="true">
> <library name="tuscany_sca_ws_reference"/>
> <library name="tuscany_sca_ws_service"/>
> </scacppp:extension>
>
> So the package would look like:
>
> extensions/
> ws/
> tusany_sca_ws_binding.extension
> lib/
> xsd/
> other_folder/
> ...
>
> The .extension configuration file is saying to load the library which is
> located somewhere in the package... the runtime will find it... no
> need to
> specify a path.
>
> Taking this further the configuration file could list the schema to be
> loaded. Currently the runtime will just load any it finds but these
> may not
> be needed by the runtime e.g. the schema may be for some extension
> implementation specific purpose.
>
> I think it would also be good for the extension initialization()
> method to
> take as a parameter the root of the extension e.g.
> extension("/tuscany/extensions/ws"). This would allow the extension
> package
> to contain any configuration information that it needs.
>
> I'd like to start by at least introducing the .extension file for each
> extension and loading only the specified library(ies) if the extension is
> enabled.
>
> Any thoughts?
>
>
>
Two thoughts:
- convention over configuration
- the runtime should be consumable without having to go tweak XML
configuration files
If I remember correctly, renaming the dlls to .disabled was a last
minute change to work around DLL loading errors with our M2 release on
Windows.
I agree that we should do better than renaming to .disabled, but I'd
like to understand better the actual issues that we faced before
inventing yet another XML based runtime configuration language :)
I'm aware of the following issues:
1. We need the runtime to load extension libs only, not other libs under
the extension directory which are not actual extensions
2. Same for XSDs, we need to load XSDs that contribute to SCDL, and
leave other XSDs under the extension directory alone
3. Extensions that cannot be loaded because some of their dependencies
are not there should no break the runtime
4. The system admin / installer should be able to disable extensions
that won't load because their dependencies are not there (I'm not yet
convinced that this is still an issue if we manage to solve issue #3)
Did we run into any other big issues?
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org