Shindig gadget-and-container framework

[Michael Hermanto <mh...@google.com>]

Hi all,

In Google, we've been developing a lightweight gadget-and-container
framework, called "common container", to 1) simplify container and gadget
integration model, and 2) provide near-zero barrier to entry to become a
gadget container. The framework is a combination of JS (which container
clients script source) and API RPC endpoints (which provides the JS with
runtime metadata/information). Many of the features have been implemented,
and are currently being productionized. Several Google containers are busy
prototyping (to integrate with common container) with some success and
speed.

Meanwhile, there has been an independent community effort to modernize the
Shindig container. It has a similar/same set of goals. In the spirit of
re-use, we would like to bring non-Google-specific work that we've done in
Google to Shindig. Engineers at Google and Paul Lindner have met, discussed
and agreed on the motivations and feature sets. Roughly speaking, they are
as follow --

1) JS is served via the gadgets server, as a container feature, ie:
/gadgets/js/shindig.container?c=1. This will leverage gadget server
compilation of JS and management of library dependencies through
transitive-closure.

2) Standard JS API to get metadata to render a gadget, ie:
/api/rpc?method=gadgets.get.
- API is implemented using an RPC call similar to other APIs (people,
activities, etc) in Shindig.
- RPC will return a URL template and gadget metadata, sufficient to generate
iframe render URL. Also, the response can be cached by the clientto evaluate
URL template with data for subsequent/similar requests (ie: different user
preferences) without requiring an HTTP fetch.
- Also need JS API to refresh a security (/OAuth) token, ie: /api/rpc?
method=tokens.get. Gadgets requiring tokens, both short- and long-lived,
relies on the container to have these primed in a timely fashion.

3) Standard JS API to render/navigate gadget.
- Base API renders into an HTML element, ie:
container.navigateGadget(string, Element)
- Ideally support double buffering for switching gadgets views, ie:
gadget.views.requestNavigateTo().
- Need a standard "ready-to-draw" RPC callback, when the gadget has finished
initial rendering.
- Respects preferred width, height and other gadget-specified metadata used
in rendering.
- Need rendering "styles" to show gadget in a modal dialog, show hovering
next to an HTML element, etc.
- Need concept of a "primary" gadget on the page. Navigating to this gadget
should fire a callback on the parent page to support navigation in the
browser bar.

4) All RPC APIs (gadgets, people, activities) should be callable form the
container page.
- Likely implementation is an iframe on the domain of the gadget host. Web
site calls gadgets.rpc() to iframe and iframe makes XHR to gadget host.
- Gadget render iframe URLs can be enhanced for latency-optimizing features,
ie: container can be an RPC hub for gadgets on the page.
- RPC responses can be obtained from pre-loaded/cache metadata for latency
improvements.
- Need to flesh out a standard API to redirect to a login page from the 3rd
party site.

What's next? Discuss. We would very much like to hear your thoughts. In the
next few days, we will prepare common container code base for public
viewing, and upon no strong objections, we will send an initial code drop
for review, where which we can discuss further in greater technical details.

Cheers,
--Michael

Re: Shindig gadget-and-container framework

[Michael Hermanto <mh...@google.com>]

On Wed, May 12, 2010 at 10:55 AM, Paul Lindner <pl...@linkedin.com>wrote:

> Thanks for getting this process going Michael.  This will really make
> Shindig much easier to use.
>
> Regarding some questions raised below here's my opinions:
>
> * Metadata 'render' api should allow an array of [gadget url, moduleId,
> view]
>

Yes, this is what common container is currently doing, and we will maintain
this.


> * Metadata api should be flexible enough to form the basis of a gadget
> directory
>

Many of the APIs work with JSONs. I think this should be flexible enough for
other purposes.


> * I want to see metadata and social calls batchable
> * Note that with recent checkins we can support fetching JSON-RPC results
> with JSONP or CORS.
> * Login popup should be based on OAuth 2.0 User Agent flow, which will
> allow
> for easier integration once OAuth 2.0 becomes more widespread.
>

All noted. Thanks Paul.


>
> On Wed, May 12, 2010 at 12:31 AM, Michael Hermanto <mhermanto@google.com
> >wrote:
>
> > Hi all,
> >
> > In Google, we've been developing a lightweight gadget-and-container
> > framework, called "common container", to 1) simplify container and gadget
> > integration model, and 2) provide near-zero barrier to entry to become a
> > gadget container. The framework is a combination of JS (which container
> > clients script source) and API RPC endpoints (which provides the JS with
> > runtime metadata/information). Many of the features have been
> implemented,
> > and are currently being productionized. Several Google containers are
> busy
> > prototyping (to integrate with common container) with some success and
> > speed.
> >
> > Meanwhile, there has been an independent community effort to modernize
> the
> > Shindig container. It has a similar/same set of goals. In the spirit of
> > re-use, we would like to bring non-Google-specific work that we've done
> in
> > Google to Shindig. Engineers at Google and Paul Lindner have met,
> discussed
> > and agreed on the motivations and feature sets. Roughly speaking, they
> are
> > as follow --
> >
> > 1) JS is served via the gadgets server, as a container feature, ie:
> > /gadgets/js/shindig.container?c=1. This will leverage gadget server
> > compilation of JS and management of library dependencies through
> > transitive-closure.
> >
> > 2) Standard JS API to get metadata to render a gadget, ie:
> > /api/rpc?method=gadgets.get.
> > - API is implemented using an RPC call similar to other APIs (people,
> > activities, etc) in Shindig.
> > - RPC will return a URL template and gadget metadata, sufficient to
> > generate
> > iframe render URL. Also, the response can be cached by the clientto
> > evaluate
> > URL template with data for subsequent/similar requests (ie: different
> user
> > preferences) without requiring an HTTP fetch.
> > - Also need JS API to refresh a security (/OAuth) token, ie: /api/rpc?
> > method=tokens.get. Gadgets requiring tokens, both short- and long-lived,
> > relies on the container to have these primed in a timely fashion.
> >
> > 3) Standard JS API to render/navigate gadget.
> > - Base API renders into an HTML element, ie:
> > container.navigateGadget(string, Element)
> > - Ideally support double buffering for switching gadgets views, ie:
> > gadget.views.requestNavigateTo().
> > - Need a standard "ready-to-draw" RPC callback, when the gadget has
> > finished
> > initial rendering.
> > - Respects preferred width, height and other gadget-specified metadata
> used
> > in rendering.
> > - Need rendering "styles" to show gadget in a modal dialog, show hovering
> > next to an HTML element, etc.
> > - Need concept of a "primary" gadget on the page. Navigating to this
> gadget
> > should fire a callback on the parent page to support navigation in the
> > browser bar.
> >
> > 4) All RPC APIs (gadgets, people, activities) should be callable form the
> > container page.
> > - Likely implementation is an iframe on the domain of the gadget host.
> Web
> > site calls gadgets.rpc() to iframe and iframe makes XHR to gadget host.
> > - Gadget render iframe URLs can be enhanced for latency-optimizing
> > features,
> > ie: container can be an RPC hub for gadgets on the page.
> > - RPC responses can be obtained from pre-loaded/cache metadata for
> latency
> > improvements.
> > - Need to flesh out a standard API to redirect to a login page from the
> 3rd
> > party site.
> >
> > What's next? Discuss. We would very much like to hear your thoughts. In
> the
> > next few days, we will prepare common container code base for public
> > viewing, and upon no strong objections, we will send an initial code drop
> > for review, where which we can discuss further in greater technical
> > details.
> >
> > Cheers,
> > --Michael
> >
>

Re: Shindig gadget-and-container framework

[Paul Lindner <pl...@linkedin.com>]

Thanks for getting this process going Michael.  This will really make
Shindig much easier to use.

Regarding some questions raised below here's my opinions:

* Metadata 'render' api should allow an array of [gadget url, moduleId,
view]
* Metadata api should be flexible enough to form the basis of a gadget
directory
* I want to see metadata and social calls batchable
* Note that with recent checkins we can support fetching JSON-RPC results
with JSONP or CORS.
* Login popup should be based on OAuth 2.0 User Agent flow, which will allow
for easier integration once OAuth 2.0 becomes more widespread.


On Wed, May 12, 2010 at 12:31 AM, Michael Hermanto <mh...@google.com>wrote:

> Hi all,
>
> In Google, we've been developing a lightweight gadget-and-container
> framework, called "common container", to 1) simplify container and gadget
> integration model, and 2) provide near-zero barrier to entry to become a
> gadget container. The framework is a combination of JS (which container
> clients script source) and API RPC endpoints (which provides the JS with
> runtime metadata/information). Many of the features have been implemented,
> and are currently being productionized. Several Google containers are busy
> prototyping (to integrate with common container) with some success and
> speed.
>
> Meanwhile, there has been an independent community effort to modernize the
> Shindig container. It has a similar/same set of goals. In the spirit of
> re-use, we would like to bring non-Google-specific work that we've done in
> Google to Shindig. Engineers at Google and Paul Lindner have met, discussed
> and agreed on the motivations and feature sets. Roughly speaking, they are
> as follow --
>
> 1) JS is served via the gadgets server, as a container feature, ie:
> /gadgets/js/shindig.container?c=1. This will leverage gadget server
> compilation of JS and management of library dependencies through
> transitive-closure.
>
> 2) Standard JS API to get metadata to render a gadget, ie:
> /api/rpc?method=gadgets.get.
> - API is implemented using an RPC call similar to other APIs (people,
> activities, etc) in Shindig.
> - RPC will return a URL template and gadget metadata, sufficient to
> generate
> iframe render URL. Also, the response can be cached by the clientto
> evaluate
> URL template with data for subsequent/similar requests (ie: different user
> preferences) without requiring an HTTP fetch.
> - Also need JS API to refresh a security (/OAuth) token, ie: /api/rpc?
> method=tokens.get. Gadgets requiring tokens, both short- and long-lived,
> relies on the container to have these primed in a timely fashion.
>
> 3) Standard JS API to render/navigate gadget.
> - Base API renders into an HTML element, ie:
> container.navigateGadget(string, Element)
> - Ideally support double buffering for switching gadgets views, ie:
> gadget.views.requestNavigateTo().
> - Need a standard "ready-to-draw" RPC callback, when the gadget has
> finished
> initial rendering.
> - Respects preferred width, height and other gadget-specified metadata used
> in rendering.
> - Need rendering "styles" to show gadget in a modal dialog, show hovering
> next to an HTML element, etc.
> - Need concept of a "primary" gadget on the page. Navigating to this gadget
> should fire a callback on the parent page to support navigation in the
> browser bar.
>
> 4) All RPC APIs (gadgets, people, activities) should be callable form the
> container page.
> - Likely implementation is an iframe on the domain of the gadget host. Web
> site calls gadgets.rpc() to iframe and iframe makes XHR to gadget host.
> - Gadget render iframe URLs can be enhanced for latency-optimizing
> features,
> ie: container can be an RPC hub for gadgets on the page.
> - RPC responses can be obtained from pre-loaded/cache metadata for latency
> improvements.
> - Need to flesh out a standard API to redirect to a login page from the 3rd
> party site.
>
> What's next? Discuss. We would very much like to hear your thoughts. In the
> next few days, we will prepare common container code base for public
> viewing, and upon no strong objections, we will send an initial code drop
> for review, where which we can discuss further in greater technical
> details.
>
> Cheers,
> --Michael
>

Re: Shindig gadget-and-container framework

[John Hjelmstad <fa...@google.com>]

Sorry for the very-delayed response. Still figured this is worth commenting.

On Wed, May 12, 2010 at 2:01 PM, Paul Lindner <pl...@linkedin.com> wrote:

> On Wed, May 12, 2010 at 11:16 AM, Kam Kasravi <kkasravi@yahoo-inc.com
> >wrote:
>
> > Will this framework be taking advantage of the features framework where
> > views would be appropriately expanded, etc?
> >
>
> Not sure..  John?
>

Kam, can you expand on this? Not sure what exactly is meant by expanding
views, in the context of common container. We do plan to propose a loose
definition of what a container-mediated feature "means" ie. which handlers a
given <Require feature="x"/> expands to and what their semantics are.


>
>
> > Also, has there been any effort to expand gadgets.rpc so that gadget <->
> > gadget or gadget broadcast would allow gadgets
> > to communicate with one another?
> >
>
> There's been some work integrating Shindig with OpenAjax.  That still
> relies
> upon a hub system.  I've been experimenting with pubsub, and it seems to
> work pretty well too.
>

The OpenAjax work is basically a much-improved pubsub feature, and when
fully proposed I suspect will meet most peoples' needs.

Direct gadget <-> gadget communication without the container being aware, on
the other hand, has seen little effort. It's technically not possible for
the most part, since any gadget desiring to communicate with another needs a
way to address it. The container controls this information. (there's a
longer discussion here, but suffice to say that any wily container could
MITM pretty easily; possible exception to carefully-constructed
window.postMessage browsers, but even then I'm dubious).


>
>
>
> > Finally is the new js taking advantage of SES or commonjs to load
> > resources?
> >
> >
> > Can you give us a reference to SES?  I'm not aware of what that is.
>
> commonjs looks interesting.  It's modules concept aligns pretty well with
> what we do with the js features and would be a more portable way of
> annotating dependencies compared to say closure-library.
>

Re: Shindig gadget-and-container framework

[๏̯͡๏ Jasvir Nagra <ja...@google.com>]

On Wed, May 12, 2010 at 2:01 PM, Paul Lindner <pl...@linkedin.com> wrote:

> On Wed, May 12, 2010 at 11:16 AM, Kam Kasravi <kkasravi@yahoo-inc.com
> >wrote:
>
> > Will this framework be taking advantage of the features framework where
> > views would be appropriately expanded, etc?
> >
>
> Not sure..  John?
>
>
> > Also, has there been any effort to expand gadgets.rpc so that gadget <->
> > gadget or gadget broadcast would allow gadgets
> > to communicate with one another?
> >
>
> There's been some work integrating Shindig with OpenAjax.  That still
> relies
> upon a hub system.  I've been experimenting with pubsub, and it seems to
> work pretty well too.
>
>
>
> > Finally is the new js taking advantage of SES or commonjs to load
> > resources?
> >
> >
> > Can you give us a reference to SES?  I'm not aware of what that is.
>
>
SES (See http://code.google.com/p/es-lab/wiki/SecureEcmaScript and more
generally http://code.google.com/p/es-lab) is a securable subset of ES5.  It
borrows features from the different securable javascript subsets and is
compatible with Caja.  By targeting ES5 strict, it is possible to provide
the same kind of controlled sharing and isolation that Caja provides without
the heavy rewriting.  SES is larger subset of javascript than Cajita, Valija
or AdSafe and since requires only verification or light rewriting of array
expressions, SES runs at full speed in browsers that support ES5 strict.

commonjs looks interesting.  It's modules concept aligns pretty well with
> what we do with the js features and would be a more portable way of
> annotating dependencies compared to say closure-library.
>

Re: Shindig gadget-and-container framework

[Paul Lindner <pl...@linkedin.com>]

On Wed, May 12, 2010 at 11:16 AM, Kam Kasravi <kk...@yahoo-inc.com>wrote:

> Will this framework be taking advantage of the features framework where
> views would be appropriately expanded, etc?
>

Not sure..  John?


> Also, has there been any effort to expand gadgets.rpc so that gadget <->
> gadget or gadget broadcast would allow gadgets
> to communicate with one another?
>

There's been some work integrating Shindig with OpenAjax.  That still relies
upon a hub system.  I've been experimenting with pubsub, and it seems to
work pretty well too.



> Finally is the new js taking advantage of SES or commonjs to load
> resources?
>
>
> Can you give us a reference to SES?  I'm not aware of what that is.

commonjs looks interesting.  It's modules concept aligns pretty well with
what we do with the js features and would be a more portable way of
annotating dependencies compared to say closure-library.

Re: Shindig gadget-and-container framework

[Kam Kasravi <kk...@yahoo-inc.com>]

Will this framework be taking advantage of the features framework where views would be appropriately expanded, etc?

Also, has there been any effort to expand gadgets.rpc so that gadget <-> gadget or gadget broadcast would allow gadgets 
to communicate with one another?

Finally is the new js taking advantage of SES or commonjs to load resources?


thanks
kam

On May 12, 2010, at 12:31 AM, Michael Hermanto wrote:

> Hi all,
> 
> In Google, we've been developing a lightweight gadget-and-container
> framework, called "common container", to 1) simplify container and gadget
> integration model, and 2) provide near-zero barrier to entry to become a
> gadget container. The framework is a combination of JS (which container
> clients script source) and API RPC endpoints (which provides the JS with
> runtime metadata/information). Many of the features have been implemented,
> and are currently being productionized. Several Google containers are busy
> prototyping (to integrate with common container) with some success and
> speed.
> 
> Meanwhile, there has been an independent community effort to modernize the
> Shindig container. It has a similar/same set of goals. In the spirit of
> re-use, we would like to bring non-Google-specific work that we've done in
> Google to Shindig. Engineers at Google and Paul Lindner have met, discussed
> and agreed on the motivations and feature sets. Roughly speaking, they are
> as follow --
> 
> 1) JS is served via the gadgets server, as a container feature, ie:
> /gadgets/js/shindig.container?c=1. This will leverage gadget server
> compilation of JS and management of library dependencies through
> transitive-closure.
> 
> 2) Standard JS API to get metadata to render a gadget, ie:
> /api/rpc?method=gadgets.get.
> - API is implemented using an RPC call similar to other APIs (people,
> activities, etc) in Shindig.
> - RPC will return a URL template and gadget metadata, sufficient to generate
> iframe render URL. Also, the response can be cached by the clientto evaluate
> URL template with data for subsequent/similar requests (ie: different user
> preferences) without requiring an HTTP fetch.
> - Also need JS API to refresh a security (/OAuth) token, ie: /api/rpc?
> method=tokens.get. Gadgets requiring tokens, both short- and long-lived,
> relies on the container to have these primed in a timely fashion.
> 
> 3) Standard JS API to render/navigate gadget.
> - Base API renders into an HTML element, ie:
> container.navigateGadget(string, Element)
> - Ideally support double buffering for switching gadgets views, ie:
> gadget.views.requestNavigateTo().
> - Need a standard "ready-to-draw" RPC callback, when the gadget has finished
> initial rendering.
> - Respects preferred width, height and other gadget-specified metadata used
> in rendering.
> - Need rendering "styles" to show gadget in a modal dialog, show hovering
> next to an HTML element, etc.
> - Need concept of a "primary" gadget on the page. Navigating to this gadget
> should fire a callback on the parent page to support navigation in the
> browser bar.
> 
> 4) All RPC APIs (gadgets, people, activities) should be callable form the
> container page.
> - Likely implementation is an iframe on the domain of the gadget host. Web
> site calls gadgets.rpc() to iframe and iframe makes XHR to gadget host.
> - Gadget render iframe URLs can be enhanced for latency-optimizing features,
> ie: container can be an RPC hub for gadgets on the page.
> - RPC responses can be obtained from pre-loaded/cache metadata for latency
> improvements.
> - Need to flesh out a standard API to redirect to a login page from the 3rd
> party site.
> 
> What's next? Discuss. We would very much like to hear your thoughts. In the
> next few days, we will prepare common container code base for public
> viewing, and upon no strong objections, we will send an initial code drop
> for review, where which we can discuss further in greater technical details.
> 
> Cheers,
> --Michael