Re: Working on data integration guide for Shindig

[Michael Papp <mk...@gmail.com>]

There are a number of issues here in terms of "integrating" existing/new
service backends with Shindig.  One issue is support for Shindig
implementations in different languages.  Since a "pure" implementation in
any language is going to require a re-write to that language, this issue is
best addressed by a discussion going on (at the moment) on another thread.
This concerns refactoring the core Java code base into groupings which make
language-specific implementations easier to approach.

To focus on the Java implementation, specifically, raises the second issue.
The cleanest approach, from my perspective, is the one alluded to on the
project page for Shindig: a Service Provider Interface.  Irregardless of the
semantics of this phrase in other languages, it has a very clear and
well-defined meaning in Java.  One should be able to, by implementing a set
of public interfaces and/or abstract classes, provide a drop-in replacement
with their implementation for the implementation that ships with Shindig.
By properly packaging a jar file and placing it on the classpath, no
'configuration' is needed to activate this new implementation ala the Java
sound SPI.  This would be my preference as it should not, if done properly,
require any changes to the Shindig code and allows one to simply focus on
providing the proper implementations to connect the existing project to
one's database and other backend infrastructure.

Dependency injection would also work well, albeit with slightly tighter
coupling to the Shindig code itself.  Not to initiate a conversation about
DI pros and cons, but changes to Shindig internals is more likely to require
one to modify or rewrite one's code with the DI approach then with a pure
SPI offering.

Lastly, the "classNames as servlet init-params in the web.xml file"
approximates the SPI approach, but that requires the project take a more
stringent approach to identifying and isolating "swappable" class units then
an SPI approach.  An SPI is a contract; this approach is basically a
promise.  To clarify, interdependencies in the class hierarchy and shared
dependencies must be observed such that swapping certain classes remain
possible as the project moves forward.

The RPC approach that is currently being promoted is also a good approach at
providing abstraction and avoiding implementation dependencies.  The 'bad'
part of this approach is that you already have all pieces of a Java
implementation that could so easily provide an SPI and hook directly to the
backend, without the need for implementing a layer of RPC services on top of
existing backend functionality.

-- Michael


From: "Kevin Brown" <et...@google.com>
To: shindig-dev@incubator.apache.org, rauge@liferay.com
Date: Wed, 2 Apr 2008 22:44:24 -0700
Subject: Re: Working on data integration guide for Shindig...
On Wed, Apr 2, 2008 at 9:20 PM, Raymond Auge <ra...@liferay.com> wrote:

> The shindig plugin we developed works as a fire and forget app in all
> the app server/servlet containers/RDBMS combinations that we support
> (which is extensive, literally hundreds of combinations).
>
> What we're missing is that "injectable" datasource backend.
>
> Right now I'm flat out replacing the implementation of
>
> org.apache.shindig.social.opensocial.OpenSocialDataHandler
>
> overriding the implementations of the Basic*Services.
>
> But that's not a very nice way of working with it. At the very least we
> need some way to wire in an implementation. This way you could have X
> backends bundled to handle the different possible scenarios, direct
> JDBC, DAO Service Tier, RPC implementation, etc, while still allowing
> for custom implementations.
>
> In our case, we're plugging in directly to our generated DAO service
> tier, so as to leverage existing caching infrastructure, cluster
> support, etc..., which means we'd want to implement the backend as a
> custom code anyway.
>
> The question is "which is the most accepted way of building out these
> plug-able backend implementations?"
>
> You can go anywhere from
> - classNames as servlet init-params in the web.xml file
> - classpath properties files specifying implementation classes
> ... all the way to ...
> - a Dependency Injection container like Spring...


The DI wiring is already started (as of about 5pm today), using Guice.
Spring or other frameworks can be layered on top of this as appropriate.

Again, though, this only satisfies the case of sites that are already Java
based.
--
~Kevin

Re: Working on data integration guide for Shindig

[Kevin Brown <et...@google.com>]

On Thu, Apr 3, 2008 at 11:17 AM, Michael Papp <mk...@gmail.com> wrote:

> Dependency injection would also work well, albeit with slightly tighter
> coupling to the Shindig code itself.  Not to initiate a conversation about
> DI pros and cons, but changes to Shindig internals is more likely to
> require
> one to modify or rewrite one's code with the DI approach then with a pure
> SPI offering.


Even with an SPI approach, there will still be DI anyway (even if it's only
used internally). Very few people will likely have to replace
BasicRemoteContentFetcher, for instance (so it doesn't belong in the SPI),
but there will be occasional deployments that require something else. DI
makes testing easier, and that's why we'll design the code in that way
regardless of its other benefits. I'd suspect that the social data pieces
are the best fit for an SPI approach.

Lastly, the "classNames as servlet init-params in the web.xml file"
> approximates the SPI approach, but that requires the project take a more
> stringent approach to identifying and isolating "swappable" class units
> then
> an SPI approach.  An SPI is a contract; this approach is basically a
> promise.  To clarify, interdependencies in the class hierarchy and shared
> dependencies must be observed such that swapping certain classes remain
> possible as the project moves forward.


This is basically spring though. Instead of reinventing it, we should just
design the code in a way that allows someone who likes spring to use spring.


The RPC approach that is currently being promoted is also a good approach at
> providing abstraction and avoiding implementation dependencies.  The 'bad'
> part of this approach is that you already have all pieces of a Java
> implementation that could so easily provide an SPI and hook directly to
> the
> backend, without the need for implementing a layer of RPC services on top
> of
> existing backend functionality.


This is likely not an issue anyway -- in practice, the pieces that need to
be wired together are best dealt with as independent servers in a production
environment for scalability. Small sites (less than a million requests per
day) can probably get away with this, but sites this small are rarely using
Java anyway so it wouldn't matter that much if it was easier to use an spi
or implement some interfaces or anything else because they can't use it in
either case.

-- 
~Kevin