Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

Hi all,

This has been cooking for a little while (I think it was about 6 month
ago that I mentioned I should put my money/code where my mouth was),
but I finally submitted a patch for the ESME-14 Jira issue
(https://issues.apache.org/jira/browse/ESME-14). This patch creates a
new branch called "new_api". The sole purpose of this branch is to
create a new API endpoint /api2/ where we can start converting the API
to a format that aligns with discussions occurring on the wiki
(http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).

The current diff has a working API at /api2/ that has pretty much the
same methods as the old one (I just copied them after all), but
arranged in the resource/object hierarchy outlined on the wiki. I've
also moved some parameters out of the query (part of the URL after the
?) and into the path of the URL, when these parameters actually refer
to resources/objects.

The diff should actually exist nicely alongside the existing API and
functionality, but I figured the easiest and safest way to get it in
initial was in a branch.

Here's what I'm looking at doing next, not necessarily in this order:

1. Create new handlers to fill out the object/resource hierarchy
2. Fix up the existing code in the API2.scala file so that it all
behaves the same way
3. Work on aligning and documenting the naming of parameters that API
handlers expect
4. Work on accepting XML or JSON representations of new or changed
resources, rather than using query parameters
5. Write tests/specs (I'm working on figuring out how to do this. Any
pointers on how to make an HTTP request from Scala/Lift would be
extremely helpful, as I think this is level at which I need to test
the API.)

See the wiki page for longer-term discussions and approaches.

Any comments or questions? I've got to note that this is the first
Scala I've ever written (though truly, it's mostly copied and adjusted
at this point) outside of examples from books, so please please please
tell me how I should be doing things differently.

Thanks,
Ethan

Re: Patch submitted, working towards API design

[David Pollak <fe...@gmail.com>]

And not to put a fine enough point on this discussion, but Novell announced
Pulse today... it's a Lift-base enterprise front-end to Google Wave.  See:
http://twitter.com/djspiewak/statuses/5424470339
http://twitter.com/djspiewak/statuses/5424501524
http://twitter.com/djspiewak/statuses/5424817863

So, if Lift and Comet are good and scalable enough for Novell to adopt for a
key project, what could possibly argue against using the technology in
ESME?  To further amplify on this, there was not a single Comet-related
ticket or issue that Novell identified in Lift as part of this project, so
I'm pretty sure that any hand-waving about scalability is pure, untested BS.



On Wed, Nov 4, 2009 at 6:35 AM, David Pollak
<fe...@gmail.com>wrote:

>
>
> On Wed, Nov 4, 2009 at 2:58 AM, Markus Kohler <ma...@gmail.com>wrote:
>
>> Hi Vassil,
>> Sorry, it was not clear to me that  real time is a key differentiator for
>> ESME, and it's already build around it.
>> I'm just saying, that you shouldn't expect that this scales easily to
>> thousands of users( maybe it doesn't need to) without special Web
>> container
>> support. That might not even be a problem AFAIK besides Jetty, Tomcat has
>> it
>> and Glassfish 3 probably has it as well.
>>
>
> Lift automatically detects that it's running in a Jetty and uses Jetty
> continuations to support long polling without consuming thread resources.
> That's the whole premise behind CometActors.  As other popular containers
> and/or standards appear that support continuations without hardcoding
> interfaces, Lift will automatically support those as well.  So, you're
> scalability argument has no grounds in technical realities.
>
> Further, if you take a look at the Lift committer team (
> http://www.liftweb.net/team.html ), you might recognize names and faces
> from a more popular micromessaging service that transitioned to using Scala
> for scalability.  Hmmm.... maybe there's something about the abstractions
> that Scala has to offer that allow for the building of scalable
> micromessaging systems.
>
> If you go back in the ESME archives, you'll see another thread similar to
> this one.
>
> I am absolutely opposed to trying to force the event based nature of ESME
> (it's an event manipulation system that's more usable by real humans than
> the likes of AMQP) into the limitations of REST and the REST concept.  REST
> is the antithesis of flowing messages.  It's modeled on static.  ESME is
> modeled on dynamic.  I appreciate the efforts and thought that Ethan has put
> into the APIs, but at the end of the day, things that treat REST and the
> primary design goal will be flushed from ESME in favor of building API
> abstractions that favor events over a static model.
>
>
>
>>
>> From the REST API point of view I don't think there's much to do, other
>> than
>> appending something to the URL that indicates that connection needs to be
>>  kept open for COMET style communication.
>>
>> One point that the REST API probably needs to support is a way for not
>> always connected clients to "sync" only new messages.
>>
>> Googles gdata does this with etags:
>>
>> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
>> ,
>> which make sense to me, at least at a first glance.
>> <
>> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
>> >
>>
>>
>> Regards,
>> Markus
>>
>> On Wed, Nov 4, 2009 at 10:05 AM, Vassil Dichev <vd...@apache.org>
>> wrote:
>>
>> > > I understand that it would be nice to show  Scala advantages, but if
>> real
>> > > time messages, such as in a chat are not the first goal, I would
>> rather
>> > > concentrate on supporting pagination. Just my humble opinion of
>> course.
>> >
>> > I've also said before that pagination is great to have, but "real time
>> > messages, such as in a chat" are not a goal, they exist in ESME right
>> > now, and ESME design revolves around that.
>> >
>> > > Oh and BTW properly (performance) testing Comet based applications
>> turned
>> > > out to be an up hill battle. I'm still convinced that I can deliver
>> > > something, but it will take more time. The issue is that "standard"
>> load
>> > > testing tools such as Loadrunner or JMeter are based on the assumption
>> > that
>> > > you just want to replay short HTTP requests. That approach doesn't
>> really
>> > > work with Comet based apps because they hold a connection open.
>> >
>> > True. That's one excuse we don't have that many tests. I'd like to
>> > think it's the major one, but unfortunately it's not.
>> >
>>
>
>
>
> --
> Lift, the simply functional web framework http://liftweb.net
> Beginning Scala http://www.apress.com/book/view/1430219890
> Follow me: http://twitter.com/dpp
> Surf the harmonics
>



-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

Hi Vassil,
I fully agree.
Regards,
Markus

On Wed, Nov 4, 2009 at 7:14 PM, Vassil Dichev <vd...@apache.org> wrote:

> OK, let's start the discussion again. For the API calls which are not
> comet-related, ESME could benefit to apply the RESTful principles
> which make sense.
>
> As for Comet, my team has rejected Yammer in the past for short
> scrum-like meetings of remote teams because if everyone has to wait
> what the others have to say, the meeting is prohibitively long (and it
> takes less than 30 seconds to update one message). I don't claim that
> everyone has this need or even that we're not trying to fit a tool
> into a scenario it wasn't meant to do. But it sure is one of the cool
> factors for everyone I've demoed ESME to. If we're not different than
> the competition- laconi.ca, Yammer, present.ly, etc., then I'm not
> sure we can compete just on the speed of copying features from the
> others.
>
> At any rate, time will tell if Comet is a key differentiator or is
> dragging us down. But since we have it already, why would we give it
> up?
>
> I would even suggest that we have both APIs compete with each other
> and see which one clients would choose to use.
>
> Vassil
>

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

Hi Ethan
I also thought about XMPP, but I think it's not really HTTP based, but using
raw sockets.

Also there seems to be ways around this. Google wave seems also to use
XMPP.

I think it would good to have an API that can be used by rich clients as
well as browser based applications(mashups?).

Regards,
Markus

On Thu, Nov 5, 2009 at 2:30 PM, Ethan Jewett <es...@gmail.com> wrote:

> Hi all,
>
> One of the key design goals of this new API is to use RESTful
> approaches for HTTP resources and Streaming approaches for
> Streaming/Message-queue resources. Which is to say, I'm trying to do
> exactly what Vassil says.
>
> So yes, I really want to do a streaming API for streams/queues, and
> I'm going to re-enable the comet-based interface in the existing API
> at the API2 endpoint just as soon as I figure out how the client
> should indicate that it wants to use Comet-based long-polling of a
> resource (which I think was what Markus was giving input on when this
> whole conversation spiraled out of control).
>
> However I don't plan on this being the long-term approach because I
> think that it is fundamentally flawed to stream over HTTP. This seems
> to be just about the least efficient way we can do this. My reading
> indicates that XMPP is probably better and AMQP is probably best. I'd
> still looking for points of view on this as I am by no means an
> expert.
>
> I'm also concerned about the approach of using the Lift comet actors
> to stream, because the implementation could (and apparently will,
> based on comments from Vassil and David) change in Lift and break the
> API. The ability to abstract the implementation details is great if
> you are building a web application where you control both the client
> (browser) and server (Pulse, for example), but we are trying to build
> an API in addition to a web interface, which (at least in my mind)
> requires a reliable interface that serves as an abstraction of the
> backend, so that every implementation change on the server doesn't
> break clients. If the server implementation of the messages API
> changes, clients are not going to change, they are just going to
> break.
>
> So anyway, those are my concerns. I'm on board with the goal, I'm just
> advocating for considering other approaches. And better, I'm now up to
> speed enough to write some code to try them out! :-)
>
> Ethan
>
> On Wed, Nov 4, 2009 at 1:14 PM, Vassil Dichev <vd...@apache.org> wrote:
> > OK, let's start the discussion again. For the API calls which are not
> > comet-related, ESME could benefit to apply the RESTful principles
> > which make sense.
> >
> > As for Comet, my team has rejected Yammer in the past for short
> > scrum-like meetings of remote teams because if everyone has to wait
> > what the others have to say, the meeting is prohibitively long (and it
> > takes less than 30 seconds to update one message). I don't claim that
> > everyone has this need or even that we're not trying to fit a tool
> > into a scenario it wasn't meant to do. But it sure is one of the cool
> > factors for everyone I've demoed ESME to. If we're not different than
> > the competition- laconi.ca, Yammer, present.ly, etc., then I'm not
> > sure we can compete just on the speed of copying features from the
> > others.
> >
> > At any rate, time will tell if Comet is a key differentiator or is
> > dragging us down. But since we have it already, why would we give it
> > up?
> >
> > I would even suggest that we have both APIs compete with each other
> > and see which one clients would choose to use.
> >
> > Vassil
> >
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

Like the idea of forcing people to use extra parameter for the call.
Means it won't be the default...

D.

On Sun, Nov 15, 2009 at 5:36 PM, Ethan Jewett <es...@gmail.com> wrote:
> I've updated the API 2.0 design page with something very similar to
> this: http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design
>
> What do people think of the example/proposal on that page? It does
> make the computationally expensive history request require the
> addition of an extra parameter.
>
> Ethan
>
> On Tue, Nov 10, 2009 at 7:10 PM, David Pollak
> <fe...@gmail.com> wrote:
>> On Fri, Nov 6, 2009 at 12:07 PM, Ethan Jewett <es...@gmail.com> wrote:
>>
>>> On Thu, Nov 5, 2009 at 6:10 PM, David Pollak
>>> <fe...@gmail.com> wrote:
>>> >
>>> > Lift's CometActors are meant as UI components, not as HTTP API
>>> components.
>>> > In fact Lift's CometActor changes behavior depending on the JavaScript
>>> > library used for the given application.  But the existing HTTP APIs (the
>>> > ones that I wrote) that support long polling do not use CometActors.
>>>  What
>>> > they do is they register themselves as a listener on the User object,
>>> just
>>> > as the CometActor does, and just as an AMQP or XMPP component could.  The
>>> > CometActors are used for the UI.  Other Actors are used for APIs.  But at
>>> > the end of the day, they're listeners that support asynchronous update
>>> > messages.
>>>
>>> Gotcha. I checked out the code and I see that now (that it isn't using
>>> the front-end comet-actor). I'm going to plug the streaming interface
>>> back into the api2/ endpoint - I think I'm going to make it use the
>>> same URL and response format as the messages resource, with some
>>> additional indicator to show that the client wants the long-poll
>>> option. Here's what I'm thinking as far as reconciling the
>>> REST/Streaming-HTTP approaches (ignore the URL pattern and the use of
>>> a query string instead of a header for the moment):
>>>
>>> Client makes "long-polling" request:
>>> GET http://esme.org/api2/messages?longpoll=T
>>>
>>>
>> How about:
>> GET http://esme.org/api2/messages // long poll
>>
>> GET http://esme.org/api2/messages?timeout=20 // 20 second timeout
>>
>> So, by default, the server chooses the request timeout, but the client can
>> modify the timeout based on its needs.
>>
>> This also makes the long polling the default, but allows you to deal with
>> other clients.  It's also not a binary thing, but a tuneable parameter.
>>
>>
>>> This request only returns when a new message shows up for the user. So
>>> what if the client wants to grab the last X messages for the user (to
>>> populate a timeline, for example)? Then (after it verifies that the
>>> long-poll is set up), it:
>>>
>>> GET http://esme.org/api2/messages
>>>
>>> This way, clients that don't need or don't support long-polling
>>> functionality don't need to use it (YQL only supports 30 seconds of
>>> processing, for example, so not a very long poll) and clients that
>>> don't need all the old messages in the mailbox can just tell the API
>>> that they want to wait for new messages.
>>>
>>> Here's an interesting question: Should the "messages" resource without
>>> long-polling return the last X messages or should it only return new
>>> messages (returning a 304 Not Modified response in the event that
>>> there are no new messages since the last request).
>>
>>
>> I think the messages call should return the set of available messages.  That
>> may be a null set.
>>
>>
>>> Should there be a
>>> way to request the last X messages regardless, in case a client fails
>>> to properly receive the response for some reason? I'm leaning towards
>>> the "new messages only with a 304 response option".
>>>
>>
>> I think having a separate history call would make sense.  It's going to be
>> more computational expensive to walk the message list for history (no, it's
>> not going to bring the system to a grinding halt with our current workload,
>> but it's conceptually different from "give me the latest stuff in the bucket
>> or hang out for a while if the bucket's empty")
>>
>>
>>>
>>> BTW, I'm not talking about using eTags here because my understanding
>>> is that those are primarily about cache-control, which is not really
>>> the point with streams. Maybe they would make sense on the resource
>>> representations we have, and I'd like to discuss how to set that up
>>> and whether Lift supports them (I'll keep my eyes open for this).
>>>
>>> Anyhow, that's my current thinking. Thoughts?
>>>
>>> > Lift's API is in fact stable and where there are occasional breaking
>>> > changes, the compiler flags the issues (e.g., moving from Scala's Actor
>>> > implementation to Lift's Actor implementation).  In no case would the
>>> change
>>> > of a Lift internal API make it impossible to support an existing HTTP API
>>> > (or an existing AMQP or XMPP API for that matter.)
>>> >
>>> > But, more importantly, Xerox, Novell, and Four Square are among the high
>>> > profile companies that have built and deployed customer-facing
>>> applications
>>> > on Lift.  Breaking APIs so massively that web APIs no longer worked would
>>> > cause these guys to come after me with a meat ax.
>>>
>>> Yup, I see that, and I'm not worried about Lift's abstractions
>>> breaking the API when they are used correctly. I was just under the
>>> mistaken impression that the current API was using the
>>> front-end-oriented LiftCometActors. Sorry about that :-(
>>>
>>> Ethan
>>>
>>
>>
>>
>> --
>> Lift, the simply functional web framework http://liftweb.net
>> Beginning Scala http://www.apress.com/book/view/1430219890
>> Follow me: http://twitter.com/dpp
>> Surf the harmonics
>>
>

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

I've updated the API 2.0 design page with something very similar to
this: http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design

What do people think of the example/proposal on that page? It does
make the computationally expensive history request require the
addition of an extra parameter.

Ethan

On Tue, Nov 10, 2009 at 7:10 PM, David Pollak
<fe...@gmail.com> wrote:
> On Fri, Nov 6, 2009 at 12:07 PM, Ethan Jewett <es...@gmail.com> wrote:
>
>> On Thu, Nov 5, 2009 at 6:10 PM, David Pollak
>> <fe...@gmail.com> wrote:
>> >
>> > Lift's CometActors are meant as UI components, not as HTTP API
>> components.
>> > In fact Lift's CometActor changes behavior depending on the JavaScript
>> > library used for the given application.  But the existing HTTP APIs (the
>> > ones that I wrote) that support long polling do not use CometActors.
>>  What
>> > they do is they register themselves as a listener on the User object,
>> just
>> > as the CometActor does, and just as an AMQP or XMPP component could.  The
>> > CometActors are used for the UI.  Other Actors are used for APIs.  But at
>> > the end of the day, they're listeners that support asynchronous update
>> > messages.
>>
>> Gotcha. I checked out the code and I see that now (that it isn't using
>> the front-end comet-actor). I'm going to plug the streaming interface
>> back into the api2/ endpoint - I think I'm going to make it use the
>> same URL and response format as the messages resource, with some
>> additional indicator to show that the client wants the long-poll
>> option. Here's what I'm thinking as far as reconciling the
>> REST/Streaming-HTTP approaches (ignore the URL pattern and the use of
>> a query string instead of a header for the moment):
>>
>> Client makes "long-polling" request:
>> GET http://esme.org/api2/messages?longpoll=T
>>
>>
> How about:
> GET http://esme.org/api2/messages // long poll
>
> GET http://esme.org/api2/messages?timeout=20 // 20 second timeout
>
> So, by default, the server chooses the request timeout, but the client can
> modify the timeout based on its needs.
>
> This also makes the long polling the default, but allows you to deal with
> other clients.  It's also not a binary thing, but a tuneable parameter.
>
>
>> This request only returns when a new message shows up for the user. So
>> what if the client wants to grab the last X messages for the user (to
>> populate a timeline, for example)? Then (after it verifies that the
>> long-poll is set up), it:
>>
>> GET http://esme.org/api2/messages
>>
>> This way, clients that don't need or don't support long-polling
>> functionality don't need to use it (YQL only supports 30 seconds of
>> processing, for example, so not a very long poll) and clients that
>> don't need all the old messages in the mailbox can just tell the API
>> that they want to wait for new messages.
>>
>> Here's an interesting question: Should the "messages" resource without
>> long-polling return the last X messages or should it only return new
>> messages (returning a 304 Not Modified response in the event that
>> there are no new messages since the last request).
>
>
> I think the messages call should return the set of available messages.  That
> may be a null set.
>
>
>> Should there be a
>> way to request the last X messages regardless, in case a client fails
>> to properly receive the response for some reason? I'm leaning towards
>> the "new messages only with a 304 response option".
>>
>
> I think having a separate history call would make sense.  It's going to be
> more computational expensive to walk the message list for history (no, it's
> not going to bring the system to a grinding halt with our current workload,
> but it's conceptually different from "give me the latest stuff in the bucket
> or hang out for a while if the bucket's empty")
>
>
>>
>> BTW, I'm not talking about using eTags here because my understanding
>> is that those are primarily about cache-control, which is not really
>> the point with streams. Maybe they would make sense on the resource
>> representations we have, and I'd like to discuss how to set that up
>> and whether Lift supports them (I'll keep my eyes open for this).
>>
>> Anyhow, that's my current thinking. Thoughts?
>>
>> > Lift's API is in fact stable and where there are occasional breaking
>> > changes, the compiler flags the issues (e.g., moving from Scala's Actor
>> > implementation to Lift's Actor implementation).  In no case would the
>> change
>> > of a Lift internal API make it impossible to support an existing HTTP API
>> > (or an existing AMQP or XMPP API for that matter.)
>> >
>> > But, more importantly, Xerox, Novell, and Four Square are among the high
>> > profile companies that have built and deployed customer-facing
>> applications
>> > on Lift.  Breaking APIs so massively that web APIs no longer worked would
>> > cause these guys to come after me with a meat ax.
>>
>> Yup, I see that, and I'm not worried about Lift's abstractions
>> breaking the API when they are used correctly. I was just under the
>> mistaken impression that the current API was using the
>> front-end-oriented LiftCometActors. Sorry about that :-(
>>
>> Ethan
>>
>
>
>
> --
> Lift, the simply functional web framework http://liftweb.net
> Beginning Scala http://www.apress.com/book/view/1430219890
> Follow me: http://twitter.com/dpp
> Surf the harmonics
>

Re: Patch submitted, working towards API design

[David Pollak <fe...@gmail.com>]

On Fri, Nov 6, 2009 at 12:07 PM, Ethan Jewett <es...@gmail.com> wrote:

> On Thu, Nov 5, 2009 at 6:10 PM, David Pollak
> <fe...@gmail.com> wrote:
> >
> > Lift's CometActors are meant as UI components, not as HTTP API
> components.
> > In fact Lift's CometActor changes behavior depending on the JavaScript
> > library used for the given application.  But the existing HTTP APIs (the
> > ones that I wrote) that support long polling do not use CometActors.
>  What
> > they do is they register themselves as a listener on the User object,
> just
> > as the CometActor does, and just as an AMQP or XMPP component could.  The
> > CometActors are used for the UI.  Other Actors are used for APIs.  But at
> > the end of the day, they're listeners that support asynchronous update
> > messages.
>
> Gotcha. I checked out the code and I see that now (that it isn't using
> the front-end comet-actor). I'm going to plug the streaming interface
> back into the api2/ endpoint - I think I'm going to make it use the
> same URL and response format as the messages resource, with some
> additional indicator to show that the client wants the long-poll
> option. Here's what I'm thinking as far as reconciling the
> REST/Streaming-HTTP approaches (ignore the URL pattern and the use of
> a query string instead of a header for the moment):
>
> Client makes "long-polling" request:
> GET http://esme.org/api2/messages?longpoll=T
>
>
How about:
GET http://esme.org/api2/messages // long poll

GET http://esme.org/api2/messages?timeout=20 // 20 second timeout

So, by default, the server chooses the request timeout, but the client can
modify the timeout based on its needs.

This also makes the long polling the default, but allows you to deal with
other clients.  It's also not a binary thing, but a tuneable parameter.


> This request only returns when a new message shows up for the user. So
> what if the client wants to grab the last X messages for the user (to
> populate a timeline, for example)? Then (after it verifies that the
> long-poll is set up), it:
>
> GET http://esme.org/api2/messages
>
> This way, clients that don't need or don't support long-polling
> functionality don't need to use it (YQL only supports 30 seconds of
> processing, for example, so not a very long poll) and clients that
> don't need all the old messages in the mailbox can just tell the API
> that they want to wait for new messages.
>
> Here's an interesting question: Should the "messages" resource without
> long-polling return the last X messages or should it only return new
> messages (returning a 304 Not Modified response in the event that
> there are no new messages since the last request).


I think the messages call should return the set of available messages.  That
may be a null set.


> Should there be a
> way to request the last X messages regardless, in case a client fails
> to properly receive the response for some reason? I'm leaning towards
> the "new messages only with a 304 response option".
>

I think having a separate history call would make sense.  It's going to be
more computational expensive to walk the message list for history (no, it's
not going to bring the system to a grinding halt with our current workload,
but it's conceptually different from "give me the latest stuff in the bucket
or hang out for a while if the bucket's empty")


>
> BTW, I'm not talking about using eTags here because my understanding
> is that those are primarily about cache-control, which is not really
> the point with streams. Maybe they would make sense on the resource
> representations we have, and I'd like to discuss how to set that up
> and whether Lift supports them (I'll keep my eyes open for this).
>
> Anyhow, that's my current thinking. Thoughts?
>
> > Lift's API is in fact stable and where there are occasional breaking
> > changes, the compiler flags the issues (e.g., moving from Scala's Actor
> > implementation to Lift's Actor implementation).  In no case would the
> change
> > of a Lift internal API make it impossible to support an existing HTTP API
> > (or an existing AMQP or XMPP API for that matter.)
> >
> > But, more importantly, Xerox, Novell, and Four Square are among the high
> > profile companies that have built and deployed customer-facing
> applications
> > on Lift.  Breaking APIs so massively that web APIs no longer worked would
> > cause these guys to come after me with a meat ax.
>
> Yup, I see that, and I'm not worried about Lift's abstractions
> breaking the API when they are used correctly. I was just under the
> mistaken impression that the current API was using the
> front-end-oriented LiftCometActors. Sorry about that :-(
>
> Ethan
>



-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

On Thu, Nov 5, 2009 at 6:10 PM, David Pollak
<fe...@gmail.com> wrote:
>
> Lift's CometActors are meant as UI components, not as HTTP API components.
> In fact Lift's CometActor changes behavior depending on the JavaScript
> library used for the given application.  But the existing HTTP APIs (the
> ones that I wrote) that support long polling do not use CometActors.  What
> they do is they register themselves as a listener on the User object, just
> as the CometActor does, and just as an AMQP or XMPP component could.  The
> CometActors are used for the UI.  Other Actors are used for APIs.  But at
> the end of the day, they're listeners that support asynchronous update
> messages.

Gotcha. I checked out the code and I see that now (that it isn't using
the front-end comet-actor). I'm going to plug the streaming interface
back into the api2/ endpoint - I think I'm going to make it use the
same URL and response format as the messages resource, with some
additional indicator to show that the client wants the long-poll
option. Here's what I'm thinking as far as reconciling the
REST/Streaming-HTTP approaches (ignore the URL pattern and the use of
a query string instead of a header for the moment):

Client makes "long-polling" request:
GET http://esme.org/api2/messages?longpoll=T

This request only returns when a new message shows up for the user. So
what if the client wants to grab the last X messages for the user (to
populate a timeline, for example)? Then (after it verifies that the
long-poll is set up), it:

GET http://esme.org/api2/messages

This way, clients that don't need or don't support long-polling
functionality don't need to use it (YQL only supports 30 seconds of
processing, for example, so not a very long poll) and clients that
don't need all the old messages in the mailbox can just tell the API
that they want to wait for new messages.

Here's an interesting question: Should the "messages" resource without
long-polling return the last X messages or should it only return new
messages (returning a 304 Not Modified response in the event that
there are no new messages since the last request). Should there be a
way to request the last X messages regardless, in case a client fails
to properly receive the response for some reason? I'm leaning towards
the "new messages only with a 304 response option".

BTW, I'm not talking about using eTags here because my understanding
is that those are primarily about cache-control, which is not really
the point with streams. Maybe they would make sense on the resource
representations we have, and I'd like to discuss how to set that up
and whether Lift supports them (I'll keep my eyes open for this).

Anyhow, that's my current thinking. Thoughts?

> Lift's API is in fact stable and where there are occasional breaking
> changes, the compiler flags the issues (e.g., moving from Scala's Actor
> implementation to Lift's Actor implementation).  In no case would the change
> of a Lift internal API make it impossible to support an existing HTTP API
> (or an existing AMQP or XMPP API for that matter.)
>
> But, more importantly, Xerox, Novell, and Four Square are among the high
> profile companies that have built and deployed customer-facing applications
> on Lift.  Breaking APIs so massively that web APIs no longer worked would
> cause these guys to come after me with a meat ax.

Yup, I see that, and I'm not worried about Lift's abstractions
breaking the API when they are used correctly. I was just under the
mistaken impression that the current API was using the
front-end-oriented LiftCometActors. Sorry about that :-(

Ethan

Re: Patch submitted, working towards API design

[David Pollak <fe...@gmail.com>]

On Thu, Nov 5, 2009 at 5:30 AM, Ethan Jewett <es...@gmail.com> wrote:

> Hi all,
>
> One of the key design goals of this new API is to use RESTful
> approaches for HTTP resources and Streaming approaches for
> Streaming/Message-queue resources. Which is to say, I'm trying to do
> exactly what Vassil says.
>

Cool.


>
> So yes, I really want to do a streaming API for streams/queues, and
> I'm going to re-enable the comet-based interface in the existing API
> at the API2 endpoint just as soon as I figure out how the client
> should indicate that it wants to use Comet-based long-polling of a
> resource (which I think was what Markus was giving input on when this
> whole conversation spiraled out of control).
>

I don't think you've embraced the streaming issue because you're using
REST-speak here.  Please think stream first, resources second (every time
you speak of a resource, I think the question "is this stable" [will this
request return the same result each time it's made] has to be answered).

In terms of the client, what is the issue to the client if it gets an answer
to a request within 1ms or 120,000 ms?  The client makes a request and the
client gets a response.

But more broadly, thinking REST-first is hugely problematic to me.  A
REST-centric view of the world leads to one design.  An event-centric view
of the world leads to another design.  Twitter had a REST-centric view of
the world and the fail-whale resulted.  Having to repeatedly fight the
RESTinista attitude leads to extreme frustration on my part.  Why?  I
approach problems differently and my approach often leads to innovative
solutions.

I am not advocating for a SOAP-WSDL-Heavy weight thingy here.  I'm all for
short, expressive URLs.  I'm all for addressing static resources as static
resources.  However, a stream of information is not a static resource.  It's
something that intentionally changes on each request.  State/session is part
of this.  Yes, the underlying HTTP requests are chunky, but the stream
that's part of a session, the chunkiness goes away.

So, I'm opinionated... the question is do my opinions have merit?  I am very
opinionated about mental models and design philosophy impacting actual
implementation... thus the harping on stream thing.

So, yes I will keep derailing the REST-first, I don't understand why
Comet/long-polling is good, Comet/long-polling is bad for performance
discussions.  Perhaps the best thing to do is to not have them anymore.
Just buy into stream-based.


>
> However I don't plan on this being the long-term approach because I
> think that it is fundamentally flawed to stream over HTTP.


Why?  Most of the Internet is optimized for HTTP.   HTTP 1.1 with keep-alive
means that the cost of TCP/IP setup-tear-down generally goes away.  The code
path for parsing and distributing an XMPP block over an open connection is
in the same order of magnitude as parsing an HTTP request made on a
keep-alive session and distributed within your application.


> This seems
> to be just about the least efficient way we can do this. My reading
> indicates that XMPP is probably better and AMQP is probably best. I'd
> still looking for points of view on this as I am by no means an
> expert.
>

Neither XMPP nor AMQP exclude HTTP long polling.  In fact, keeping with a
streaming design philosophy (rather than REST/resource oriented design
philosophy) allows a much easier connection between the internals and any of
the above technologies.


>
> I'm also concerned about the approach of using the Lift comet actors
> to stream, because the implementation could (and apparently will,
> based on comments from Vassil and David) change in Lift and break the
> API.


Lift's CometActors are meant as UI components, not as HTTP API components.
In fact Lift's CometActor changes behavior depending on the JavaScript
library used for the given application.  But the existing HTTP APIs (the
ones that I wrote) that support long polling do not use CometActors.  What
they do is they register themselves as a listener on the User object, just
as the CometActor does, and just as an AMQP or XMPP component could.  The
CometActors are used for the UI.  Other Actors are used for APIs.  But at
the end of the day, they're listeners that support asynchronous update
messages.


> The ability to abstract the implementation details is great if
> you are building a web application where you control both the client
> (browser) and server (Pulse, for example), but we are trying to build
> an API in addition to a web interface, which (at least in my mind)
> requires a reliable interface that serves as an abstraction of the
> backend,


Lift's API is in fact stable and where there are occasional breaking
changes, the compiler flags the issues (e.g., moving from Scala's Actor
implementation to Lift's Actor implementation).  In no case would the change
of a Lift internal API make it impossible to support an existing HTTP API
(or an existing AMQP or XMPP API for that matter.)

But, more importantly, Xerox, Novell, and Four Square are among the high
profile companies that have built and deployed customer-facing applications
on Lift.  Breaking APIs so massively that web APIs no longer worked would
cause these guys to come after me with a meat ax.


> so that every implementation change on the server doesn't
> break clients. If the server implementation of the messages API
> changes, clients are not going to change, they are just going to
> break.
>




>
> So anyway, those are my concerns. I'm on board with the goal, I'm just
> advocating for considering other approaches. And better, I'm now up to
> speed enough to write some code to try them out! :-)
>




>
> Ethan
>
> On Wed, Nov 4, 2009 at 1:14 PM, Vassil Dichev <vd...@apache.org> wrote:
> > OK, let's start the discussion again. For the API calls which are not
> > comet-related, ESME could benefit to apply the RESTful principles
> > which make sense.
> >
> > As for Comet, my team has rejected Yammer in the past for short
> > scrum-like meetings of remote teams because if everyone has to wait
> > what the others have to say, the meeting is prohibitively long (and it
> > takes less than 30 seconds to update one message). I don't claim that
> > everyone has this need or even that we're not trying to fit a tool
> > into a scenario it wasn't meant to do. But it sure is one of the cool
> > factors for everyone I've demoed ESME to. If we're not different than
> > the competition- laconi.ca, Yammer, present.ly, etc., then I'm not
> > sure we can compete just on the speed of copying features from the
> > others.
> >
> > At any rate, time will tell if Comet is a key differentiator or is
> > dragging us down. But since we have it already, why would we give it
> > up?
> >
> > I would even suggest that we have both APIs compete with each other
> > and see which one clients would choose to use.
> >
> > Vassil
> >
>



-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

Hi all,

One of the key design goals of this new API is to use RESTful
approaches for HTTP resources and Streaming approaches for
Streaming/Message-queue resources. Which is to say, I'm trying to do
exactly what Vassil says.

So yes, I really want to do a streaming API for streams/queues, and
I'm going to re-enable the comet-based interface in the existing API
at the API2 endpoint just as soon as I figure out how the client
should indicate that it wants to use Comet-based long-polling of a
resource (which I think was what Markus was giving input on when this
whole conversation spiraled out of control).

However I don't plan on this being the long-term approach because I
think that it is fundamentally flawed to stream over HTTP. This seems
to be just about the least efficient way we can do this. My reading
indicates that XMPP is probably better and AMQP is probably best. I'd
still looking for points of view on this as I am by no means an
expert.

I'm also concerned about the approach of using the Lift comet actors
to stream, because the implementation could (and apparently will,
based on comments from Vassil and David) change in Lift and break the
API. The ability to abstract the implementation details is great if
you are building a web application where you control both the client
(browser) and server (Pulse, for example), but we are trying to build
an API in addition to a web interface, which (at least in my mind)
requires a reliable interface that serves as an abstraction of the
backend, so that every implementation change on the server doesn't
break clients. If the server implementation of the messages API
changes, clients are not going to change, they are just going to
break.

So anyway, those are my concerns. I'm on board with the goal, I'm just
advocating for considering other approaches. And better, I'm now up to
speed enough to write some code to try them out! :-)

Ethan

On Wed, Nov 4, 2009 at 1:14 PM, Vassil Dichev <vd...@apache.org> wrote:
> OK, let's start the discussion again. For the API calls which are not
> comet-related, ESME could benefit to apply the RESTful principles
> which make sense.
>
> As for Comet, my team has rejected Yammer in the past for short
> scrum-like meetings of remote teams because if everyone has to wait
> what the others have to say, the meeting is prohibitively long (and it
> takes less than 30 seconds to update one message). I don't claim that
> everyone has this need or even that we're not trying to fit a tool
> into a scenario it wasn't meant to do. But it sure is one of the cool
> factors for everyone I've demoed ESME to. If we're not different than
> the competition- laconi.ca, Yammer, present.ly, etc., then I'm not
> sure we can compete just on the speed of copying features from the
> others.
>
> At any rate, time will tell if Comet is a key differentiator or is
> dragging us down. But since we have it already, why would we give it
> up?
>
> I would even suggest that we have both APIs compete with each other
> and see which one clients would choose to use.
>
> Vassil
>

Re: Patch submitted, working towards API design

[Vassil Dichev <vd...@apache.org>]

OK, let's start the discussion again. For the API calls which are not
comet-related, ESME could benefit to apply the RESTful principles
which make sense.

As for Comet, my team has rejected Yammer in the past for short
scrum-like meetings of remote teams because if everyone has to wait
what the others have to say, the meeting is prohibitively long (and it
takes less than 30 seconds to update one message). I don't claim that
everyone has this need or even that we're not trying to fit a tool
into a scenario it wasn't meant to do. But it sure is one of the cool
factors for everyone I've demoed ESME to. If we're not different than
the competition- laconi.ca, Yammer, present.ly, etc., then I'm not
sure we can compete just on the speed of copying features from the
others.

At any rate, time will tell if Comet is a key differentiator or is
dragging us down. But since we have it already, why would we give it
up?

I would even suggest that we have both APIs compete with each other
and see which one clients would choose to use.

Vassil

Re: Patch submitted, working towards API design

[David Pollak <fe...@gmail.com>]

On Wed, Nov 4, 2009 at 8:09 AM, Markus Kohler <ma...@gmail.com>wrote:

> Hi all,
>
> I just tried to give some feedback to the current state of the REST Api
> document, assuming that there was a consensus, that there's a need to for
> it. I'm not sure anymore, whether that's really the case.
>
> Just to make clear. I never questioned that a COMET based ESME can be
> implemented


It already is implemented.


> and can be scaled to a certain extent


Yeah, in fact you did question this.  This was pretty much the crux of your
argument as far as I read it.


> , nor did I question any
> competence here. I'm even not surprised that Lift/the Actor framework can
> handle the differences between app servers automatically.
>
> I just said it's harder (testing also requiring more effort), and that it's
> not clear to *me* whether this additional effort is worth it


Sorry you're unclear on the vision.  ESME is not the flavor of the day
REST.  It's something different: a human consumable event engine.  Yeah, now
Google Wave has a bunch of pieces that are not dissimilar from ESME (or vice
versa) in terms of blending synchronous and asynchronous conversations.
ESME is pushing boundaries rather than recycling a current paradigm.


> , for this kind
> of application,it's not chat isn't it?
>

Yes, it is chat.  It's immediate feedback.


>
> Maybe I didn't express it clearly but I also think that for the number of
> users (not millions I guess) that an ESME server might not need to handle
> it
> might even work without too much effort.
>

You're wrong.  Continuing to make this assertion does not serve anyone's
goals.


>
> In any case I believe that *some* Rest is still needed, if the web
> application should support bookmarking and indexing by a search engine's
> web
> crawler.
>

Beginning the design from an event basis and then deciding which APIs more
accurately reflect "static" RESTful content is the approach I've
recommended.


> It's not black and white after all.
>

No, it's not, but taking your cues from Vassil about direction rather than
disagreeing based on technology that you do not understand does not help.


>
> Regarding REST principles not compatible with an application like ESME,
> Fielding itself doesn't seem to believe that:
> http://roy.gbiv.com/untangled/2008/paper-tigers-and-hidden-dragons
>
>
He's addressing a different problem.  First, he's not addressing the
immediate feedback problem.  If users have to wait 30 seconds or a minute
for an update, there's no conversation.  Second, keeping delta timeline
queues since a certain time is expensive in terms of time and space.  While
the current ESME implementation could handle it, it does not scale.
Fielding is addressing a single queue problem (what has changed globally)
rather than a what has changed for a given user since a certain time.


> It's also not clear to me why it has to be COMET and why this cannot be
> abstracted away.


If you don't understand this then you should not be participating in this
conversation.

David


> I think a client library for a specific programming
> language could be written, that would abstract a way from where "Events"
> would come, e.g. it would be hided whether it's COMET,polling ,XMPP or
> whatever.
>
>
> Regards,
> Markus
>
> On Wed, Nov 4, 2009 at 3:35 PM, David Pollak
> <fe...@gmail.com>wrote:
>
> > On Wed, Nov 4, 2009 at 2:58 AM, Markus Kohler <markus.kohler@gmail.com
> > >wrote:
> >
> > > Hi Vassil,
> > > Sorry, it was not clear to me that  real time is a key differentiator
> for
> > > ESME, and it's already build around it.
> > > I'm just saying, that you shouldn't expect that this scales easily to
> > > thousands of users( maybe it doesn't need to) without special Web
> > container
> > > support. That might not even be a problem AFAIK besides Jetty, Tomcat
> has
> > > it
> > > and Glassfish 3 probably has it as well.
> > >
> >
> > Lift automatically detects that it's running in a Jetty and uses Jetty
> > continuations to support long polling without consuming thread resources.
> > That's the whole premise behind CometActors.  As other popular containers
> > and/or standards appear that support continuations without hardcoding
> > interfaces, Lift will automatically support those as well.  So, you're
> > scalability argument has no grounds in technical realities.
> >
> > Further, if you take a look at the Lift committer team (
> > http://www.liftweb.net/team.html ), you might recognize names and faces
> > from
> > a more popular micromessaging service that transitioned to using Scala
> for
> > scalability.  Hmmm.... maybe there's something about the abstractions
> that
> > Scala has to offer that allow for the building of scalable micromessaging
> > systems.
> >
> > If you go back in the ESME archives, you'll see another thread similar to
> > this one.
> >
> > I am absolutely opposed to trying to force the event based nature of ESME
> > (it's an event manipulation system that's more usable by real humans than
> > the likes of AMQP) into the limitations of REST and the REST concept.
>  REST
> > is the antithesis of flowing messages.  It's modeled on static.  ESME is
> > modeled on dynamic.  I appreciate the efforts and thought that Ethan has
> > put
> > into the APIs, but at the end of the day, things that treat REST and the
> > primary design goal will be flushed from ESME in favor of building API
> > abstractions that favor events over a static model.
> >
> >
> >
> > >
> > > From the REST API point of view I don't think there's much to do, other
> > > than
> > > appending something to the URL that indicates that connection needs to
> be
> > >  kept open for COMET style communication.
> > >
> > > One point that the REST API probably needs to support is a way for not
> > > always connected clients to "sync" only new messages.
> > >
> > > Googles gdata does this with etags:
> > >
> > >
> >
> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
> > > ,
> > > which make sense to me, at least at a first glance.
> > > <
> > >
> >
> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
> > > >
> > >
> > >
> > > Regards,
> > > Markus
> > >
> > > On Wed, Nov 4, 2009 at 10:05 AM, Vassil Dichev <vd...@apache.org>
> > wrote:
> > >
> > > > > I understand that it would be nice to show  Scala advantages, but
> if
> > > real
> > > > > time messages, such as in a chat are not the first goal, I would
> > rather
> > > > > concentrate on supporting pagination. Just my humble opinion of
> > course.
> > > >
> > > > I've also said before that pagination is great to have, but "real
> time
> > > > messages, such as in a chat" are not a goal, they exist in ESME right
> > > > now, and ESME design revolves around that.
> > > >
> > > > > Oh and BTW properly (performance) testing Comet based applications
> > > turned
> > > > > out to be an up hill battle. I'm still convinced that I can deliver
> > > > > something, but it will take more time. The issue is that "standard"
> > > load
> > > > > testing tools such as Loadrunner or JMeter are based on the
> > assumption
> > > > that
> > > > > you just want to replay short HTTP requests. That approach doesn't
> > > really
> > > > > work with Comet based apps because they hold a connection open.
> > > >
> > > > True. That's one excuse we don't have that many tests. I'd like to
> > > > think it's the major one, but unfortunately it's not.
> > > >
> > >
> >
> >
> >
> > --
> > Lift, the simply functional web framework http://liftweb.net
> > Beginning Scala http://www.apress.com/book/view/1430219890
> > Follow me: http://twitter.com/dpp
> > Surf the harmonics
> >
>



-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

Hi all,

I just tried to give some feedback to the current state of the REST Api
document, assuming that there was a consensus, that there's a need to for
it. I'm not sure anymore, whether that's really the case.

Just to make clear. I never questioned that a COMET based ESME can be
implemented and can be scaled to a certain extent, nor did I question any
competence here. I'm even not surprised that Lift/the Actor framework can
handle the differences between app servers automatically.

I just said it's harder (testing also requiring more effort), and that it's
not clear to *me* whether this additional effort is worth it, for this kind
of application,it's not chat isn't it?

Maybe I didn't express it clearly but I also think that for the number of
users (not millions I guess) that an ESME server might not need to handle it
might even work without too much effort.

In any case I believe that *some* Rest is still needed, if the web
application should support bookmarking and indexing by a search engine's web
crawler.
It's not black and white after all.

Regarding REST principles not compatible with an application like ESME,
Fielding itself doesn't seem to believe that:
http://roy.gbiv.com/untangled/2008/paper-tigers-and-hidden-dragons

It's also not clear to me why it has to be COMET and why this cannot be
abstracted away. I think a client library for a specific programming
language could be written, that would abstract a way from where "Events"
would come, e.g. it would be hided whether it's COMET,polling ,XMPP or
whatever.


Regards,
Markus

On Wed, Nov 4, 2009 at 3:35 PM, David Pollak
<fe...@gmail.com>wrote:

> On Wed, Nov 4, 2009 at 2:58 AM, Markus Kohler <markus.kohler@gmail.com
> >wrote:
>
> > Hi Vassil,
> > Sorry, it was not clear to me that  real time is a key differentiator for
> > ESME, and it's already build around it.
> > I'm just saying, that you shouldn't expect that this scales easily to
> > thousands of users( maybe it doesn't need to) without special Web
> container
> > support. That might not even be a problem AFAIK besides Jetty, Tomcat has
> > it
> > and Glassfish 3 probably has it as well.
> >
>
> Lift automatically detects that it's running in a Jetty and uses Jetty
> continuations to support long polling without consuming thread resources.
> That's the whole premise behind CometActors.  As other popular containers
> and/or standards appear that support continuations without hardcoding
> interfaces, Lift will automatically support those as well.  So, you're
> scalability argument has no grounds in technical realities.
>
> Further, if you take a look at the Lift committer team (
> http://www.liftweb.net/team.html ), you might recognize names and faces
> from
> a more popular micromessaging service that transitioned to using Scala for
> scalability.  Hmmm.... maybe there's something about the abstractions that
> Scala has to offer that allow for the building of scalable micromessaging
> systems.
>
> If you go back in the ESME archives, you'll see another thread similar to
> this one.
>
> I am absolutely opposed to trying to force the event based nature of ESME
> (it's an event manipulation system that's more usable by real humans than
> the likes of AMQP) into the limitations of REST and the REST concept.  REST
> is the antithesis of flowing messages.  It's modeled on static.  ESME is
> modeled on dynamic.  I appreciate the efforts and thought that Ethan has
> put
> into the APIs, but at the end of the day, things that treat REST and the
> primary design goal will be flushed from ESME in favor of building API
> abstractions that favor events over a static model.
>
>
>
> >
> > From the REST API point of view I don't think there's much to do, other
> > than
> > appending something to the URL that indicates that connection needs to be
> >  kept open for COMET style communication.
> >
> > One point that the REST API probably needs to support is a way for not
> > always connected clients to "sync" only new messages.
> >
> > Googles gdata does this with etags:
> >
> >
> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
> > ,
> > which make sense to me, at least at a first glance.
> > <
> >
> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
> > >
> >
> >
> > Regards,
> > Markus
> >
> > On Wed, Nov 4, 2009 at 10:05 AM, Vassil Dichev <vd...@apache.org>
> wrote:
> >
> > > > I understand that it would be nice to show  Scala advantages, but if
> > real
> > > > time messages, such as in a chat are not the first goal, I would
> rather
> > > > concentrate on supporting pagination. Just my humble opinion of
> course.
> > >
> > > I've also said before that pagination is great to have, but "real time
> > > messages, such as in a chat" are not a goal, they exist in ESME right
> > > now, and ESME design revolves around that.
> > >
> > > > Oh and BTW properly (performance) testing Comet based applications
> > turned
> > > > out to be an up hill battle. I'm still convinced that I can deliver
> > > > something, but it will take more time. The issue is that "standard"
> > load
> > > > testing tools such as Loadrunner or JMeter are based on the
> assumption
> > > that
> > > > you just want to replay short HTTP requests. That approach doesn't
> > really
> > > > work with Comet based apps because they hold a connection open.
> > >
> > > True. That's one excuse we don't have that many tests. I'd like to
> > > think it's the major one, but unfortunately it's not.
> > >
> >
>
>
>
> --
> Lift, the simply functional web framework http://liftweb.net
> Beginning Scala http://www.apress.com/book/view/1430219890
> Follow me: http://twitter.com/dpp
> Surf the harmonics
>

Re: Patch submitted, working towards API design

[David Pollak <fe...@gmail.com>]

On Wed, Nov 4, 2009 at 2:58 AM, Markus Kohler <ma...@gmail.com>wrote:

> Hi Vassil,
> Sorry, it was not clear to me that  real time is a key differentiator for
> ESME, and it's already build around it.
> I'm just saying, that you shouldn't expect that this scales easily to
> thousands of users( maybe it doesn't need to) without special Web container
> support. That might not even be a problem AFAIK besides Jetty, Tomcat has
> it
> and Glassfish 3 probably has it as well.
>

Lift automatically detects that it's running in a Jetty and uses Jetty
continuations to support long polling without consuming thread resources.
That's the whole premise behind CometActors.  As other popular containers
and/or standards appear that support continuations without hardcoding
interfaces, Lift will automatically support those as well.  So, you're
scalability argument has no grounds in technical realities.

Further, if you take a look at the Lift committer team (
http://www.liftweb.net/team.html ), you might recognize names and faces from
a more popular micromessaging service that transitioned to using Scala for
scalability.  Hmmm.... maybe there's something about the abstractions that
Scala has to offer that allow for the building of scalable micromessaging
systems.

If you go back in the ESME archives, you'll see another thread similar to
this one.

I am absolutely opposed to trying to force the event based nature of ESME
(it's an event manipulation system that's more usable by real humans than
the likes of AMQP) into the limitations of REST and the REST concept.  REST
is the antithesis of flowing messages.  It's modeled on static.  ESME is
modeled on dynamic.  I appreciate the efforts and thought that Ethan has put
into the APIs, but at the end of the day, things that treat REST and the
primary design goal will be flushed from ESME in favor of building API
abstractions that favor events over a static model.



>
> From the REST API point of view I don't think there's much to do, other
> than
> appending something to the URL that indicates that connection needs to be
>  kept open for COMET style communication.
>
> One point that the REST API probably needs to support is a way for not
> always connected clients to "sync" only new messages.
>
> Googles gdata does this with etags:
>
> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
> ,
> which make sense to me, at least at a first glance.
> <
> http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning
> >
>
>
> Regards,
> Markus
>
> On Wed, Nov 4, 2009 at 10:05 AM, Vassil Dichev <vd...@apache.org> wrote:
>
> > > I understand that it would be nice to show  Scala advantages, but if
> real
> > > time messages, such as in a chat are not the first goal, I would rather
> > > concentrate on supporting pagination. Just my humble opinion of course.
> >
> > I've also said before that pagination is great to have, but "real time
> > messages, such as in a chat" are not a goal, they exist in ESME right
> > now, and ESME design revolves around that.
> >
> > > Oh and BTW properly (performance) testing Comet based applications
> turned
> > > out to be an up hill battle. I'm still convinced that I can deliver
> > > something, but it will take more time. The issue is that "standard"
> load
> > > testing tools such as Loadrunner or JMeter are based on the assumption
> > that
> > > you just want to replay short HTTP requests. That approach doesn't
> really
> > > work with Comet based apps because they hold a connection open.
> >
> > True. That's one excuse we don't have that many tests. I'd like to
> > think it's the major one, but unfortunately it's not.
> >
>



-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

Hi Vassil,
Sorry, it was not clear to me that  real time is a key differentiator for
ESME, and it's already build around it.
I'm just saying, that you shouldn't expect that this scales easily to
thousands of users( maybe it doesn't need to) without special Web container
support. That might not even be a problem AFAIK besides Jetty, Tomcat has it
and Glassfish 3 probably has it as well.

>From the REST API point of view I don't think there's much to do, other than
appending something to the URL that indicates that connection needs to be
 kept open for COMET style communication.

One point that the REST API probably needs to support is a way for not
always connected clients to "sync" only new messages.

Googles gdata does this with etags:
http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning,
which make sense to me, at least at a first glance.
<http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning>


Regards,
Markus

On Wed, Nov 4, 2009 at 10:05 AM, Vassil Dichev <vd...@apache.org> wrote:

> > I understand that it would be nice to show  Scala advantages, but if real
> > time messages, such as in a chat are not the first goal, I would rather
> > concentrate on supporting pagination. Just my humble opinion of course.
>
> I've also said before that pagination is great to have, but "real time
> messages, such as in a chat" are not a goal, they exist in ESME right
> now, and ESME design revolves around that.
>
> > Oh and BTW properly (performance) testing Comet based applications turned
> > out to be an up hill battle. I'm still convinced that I can deliver
> > something, but it will take more time. The issue is that "standard" load
> > testing tools such as Loadrunner or JMeter are based on the assumption
> that
> > you just want to replay short HTTP requests. That approach doesn't really
> > work with Comet based apps because they hold a connection open.
>
> True. That's one excuse we don't have that many tests. I'd like to
> think it's the major one, but unfortunately it's not.
>

Re: Patch submitted, working towards API design

[Vassil Dichev <vd...@apache.org>]

> I understand that it would be nice to show  Scala advantages, but if real
> time messages, such as in a chat are not the first goal, I would rather
> concentrate on supporting pagination. Just my humble opinion of course.

I've also said before that pagination is great to have, but "real time
messages, such as in a chat" are not a goal, they exist in ESME right
now, and ESME design revolves around that.

> Oh and BTW properly (performance) testing Comet based applications turned
> out to be an up hill battle. I'm still convinced that I can deliver
> something, but it will take more time. The issue is that "standard" load
> testing tools such as Loadrunner or JMeter are based on the assumption that
> you just want to replay short HTTP requests. That approach doesn't really
> work with Comet based apps because they hold a connection open.

True. That's one excuse we don't have that many tests. I'd like to
think it's the major one, but unfortunately it's not.

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

Hi Vasil,
Yes. Scala Actors can certainly abstract away the implementation details,
but I assume it would still rely on the Servlet API. The features
for efficiently supporting Comet are coming with version 3.0, and that is
not yet implemented in most J2EE containers. Abstracting away whether the
client needs to do polling or uses Comet could probably be done, but would
require the use of a client library. I guess that is currently not in the
focus.

I understand that it would be nice to show  Scala advantages, but if real
time messages, such as in a chat are not the first goal, I would rather
concentrate on supporting pagination. Just my humble opinion of course.

Oh and BTW properly (performance) testing Comet based applications turned
out to be an up hill battle. I'm still convinced that I can deliver
something, but it will take more time. The issue is that "standard" load
testing tools such as Loadrunner or JMeter are based on the assumption that
you just want to replay short HTTP requests. That approach doesn't really
work with Comet based apps because they hold a connection open.

Regards,
Markus

On Wed, Nov 4, 2009 at 8:28 AM, Vassil Dichev <vd...@apache.org> wrote:

> > - "Streaming" as far as I understand is about "push" functionality.
> > I would not go as far as to say "polling is bad". Since twitter is not
> real
> > time ESME would maybe not need to be real time as well. This means one
> could
> > poll infrequently (every few minutes?) using conditional GET's, which
> would
> > not be that inefficient. Comet has the disadvantage that without special
> > server support (available in Jetty, but not in Netweaver CE for example)
>  it
> > causes a lot of threads to be created on the server, which is also not
> > really efficient.
>
> Just wanted to comment on threading- the advantage of using Scala is
> that the actor model is not necessarily associated with one thread per
> request. When using "react", there doesn't have to be a thread backing
> each actor. Also, using Lift's CometActor is a nice abstraction which
> I can imagine could even transparently implement HTML 5 WebSockets one
> day without significant changes to the client code.
>

Re: Patch submitted, working towards API design

[Vassil Dichev <vd...@apache.org>]

> - "Streaming" as far as I understand is about "push" functionality.
> I would not go as far as to say "polling is bad". Since twitter is not real
> time ESME would maybe not need to be real time as well. This means one could
> poll infrequently (every few minutes?) using conditional GET's, which would
> not be that inefficient. Comet has the disadvantage that without special
> server support (available in Jetty, but not in Netweaver CE for example)  it
> causes a lot of threads to be created on the server, which is also not
> really efficient.

Just wanted to comment on threading- the advantage of using Scala is
that the actor model is not necessarily associated with one thread per
request. When using "react", there doesn't have to be a thread backing
each actor. Also, using Lift's CometActor is a nice abstraction which
I can imagine could even transparently implement HTML 5 WebSockets one
day without significant changes to the client code.

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

On Tue, Nov 3, 2009 at 9:38 PM, Ethan Jewett <es...@gmail.com> wrote:

> Hi Markus,
>
> Wow, thank you for all the comments. I really appreciate it.


You are welcome. Note that I would not consider me the absolute expert on
this topic. I studied a lot of RESTful API's lately, because I had to help
to define several ones, but I wouldn't say that I know exactly what works in
practice and where one can make comprises.


> Responses
> and confirmations are inline:
>
> On Tue, Nov 3, 2009 at 1:21 PM, Markus Kohler <ma...@gmail.com>
> wrote:
> > - it's not 100% clear to me what you mean with
> > api2/user/tracks
> > Do you mean api2/{USERID}/tracks where {USERID} is replaced by a valid
> > USERID?
> > or do you mean that depending on the authenticated user one would get
> back
> > different results?
> > I would not recommend to use the second approach, because that means you
> > cannot bookmark the URLs anymore.
>
> I mean the later (gets the tracks of the logged in user). In my mind
> the true REST resource would be api2/users/USERID/tracks, and I would
> be open to going back to this.
>

I think I would rather go back to api2user/USERID/tracks. I would even
consider to just have only
api2user/User/USERID return a list of URLs (HATEOS) looking like
api2/Track/trackID.



> With regards to bookmarking, I do not see that as use-case we should
> support in the API. What do you have in mind there?
>
> I was talking about bookmarking, because that is feature that you get "for
free" with a proper RESTful api.
Imagine someone wants to send a link to a particular message to someone else
then it's certainly usefull for the application to be able to now how to get
the data for that particular message. Of course It could happen that the
user user does not have the access rights to see the message, but that is a
completely orthogonal issue.


> > -Regarding PUT and DELETE not available on all clients one option would
> be
> > to use google's approach in gdata and put an additional header into a
> POST
> > request:
> >
> > X-HTTP-Method-Override: PUT
>
> I like it. However, more and more HTTP client libraries are starting
> to support PUT and DELETE requests, so I'd like to wait until there is
> a demand before implementing.
>
Agreed. Flex does not support it, also there are complicated ways around it.


>
> > - as already commented  api2/session is not really Restful. I haven't
> read
> > the details about why it's currently done this way, but wouldn't simple
> HTTP
> > basic authentication work, at least if user and password are given?
>
> Agreed that sessions aren't RESTful. On the other hand, there are some
> backend optimizations that apparently benefit from sessions. It still
> exists for two reasons:
>
> 1. It works currently and I'd like to keep the API working as I make
> changes to it. I don't currently have another working method of
> managing logins.
> 2. I don't yet understand the backend optimizations that this allows
> and I'd like to understand that before ripping it out. Specifically,
> the creation of a session currently starts up an Actor that I think is
> used elsewhere in the API. I'm looking into this.
>
> With regards to basic authentication, I don't want to allow the use of
> basic authentication for the API. In general, token-based
> authentication is a good idea for APIs as it makes explicit the
> delegation that is going on when you allow an API client to act on
> your behalf. There are things I would like to change about the way the
> API uses the token to do authentication (either require HTTPS or don't
> send the token in the clear, for starters), but I'm waiting on that
> for the moment.
>
OK makes sense to me. Anyone has already looked whether SAML would be an
option here? It's becoming en vogue in Enterprise applications.

>
> > - "Streaming" as far as I understand is about "push" functionality.
> > I would not go as far as to say "polling is bad". Since twitter is not
> real
> > time ESME would maybe not need to be real time as well. This means one
> could
> > poll infrequently (every few minutes?) using conditional GET's, which
> would
> > not be that inefficient. Comet has the disadvantage that without special
> > server support (available in Jetty, but not in Netweaver CE for example)
>  it
> > causes a lot of threads to be created on the server, which is also not
> > really efficient.
> > IMHO defining a pagination protocol would be more important.
>
> Agreed, sort of. I'm defining API methods that will allow for pulling
> down a list of messages or a specific message however, I agree with
> David that we should provide better ways to access messages. Comet is
> not currently turned on in the api2/ endpoint, but I was planning to
> make it an option, though the server issue you bring up gives me
> pause. What I'd really like to do is expose each message queue to
> clients using AMQP (though what version I don't know).
>
> I don't really know what the best approach is here, so I'm looking for
> input.
>
> > In general I think it makes sense to define the URL pattern, but one has
> > always to be careful to not fall into the trap and forget about HATEOS (
> > http://blogs.sun.com/craigmcc/entry/why_hateoas).
> > With proper HATEOS support, the exact URL's would be less important, and
> > evolving the API  would be easier.
>
> I've been keeping an eye on the SunCloud API development via Tim
> Bray's blog but hadn't seen this angle. I'll have to think about it.
> How are you thinking that this could be applied to the ESME API?
>
See above. We should carefully avoid URL's that require too much knowledge
by the client.
For example api2/users/USERID/tracks looks like that the client knows that
to get the tracks for a user
the String "tracks" has to appended to the URL for the user. That's not
really "rResty".
Ok there's the argument that you might have USERID only and you would want
to jump to the tracks directly (would violate HATEOS), but because caching
can be used for most important User information,  this is not such a big
advantage (maybe I'm too vague here).

BTW, I would also definitely use Mime types.

Oh, and implementation wise, is there any technical (or legal) reason to not
use JAX-RS?
I think it should also work for Scala. With JAX-RS you could easily support
XML as well as JSON as well as other formats (I have seen a prototype of
Jersey supporting google protocol buffers) .



> Again, thank you for your feedback. I'll start incorporating it into the
> design!
>
It's my pleasure,
Greetings,
Markus

>
> Ethan
>

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

Hi Markus,

Wow, thank you for all the comments. I really appreciate it. Responses
and confirmations are inline:

On Tue, Nov 3, 2009 at 1:21 PM, Markus Kohler <ma...@gmail.com> wrote:
> - it's not 100% clear to me what you mean with
> api2/user/tracks
> Do you mean api2/{USERID}/tracks where {USERID} is replaced by a valid
> USERID?
> or do you mean that depending on the authenticated user one would get back
> different results?
> I would not recommend to use the second approach, because that means you
> cannot bookmark the URLs anymore.

I mean the later (gets the tracks of the logged in user). In my mind
the true REST resource would be api2/users/USERID/tracks, and I would
be open to going back to this.

With regards to bookmarking, I do not see that as use-case we should
support in the API. What do you have in mind there?

> -Regarding PUT and DELETE not available on all clients one option would be
> to use google's approach in gdata and put an additional header into a POST
> request:
>
> X-HTTP-Method-Override: PUT

I like it. However, more and more HTTP client libraries are starting
to support PUT and DELETE requests, so I'd like to wait until there is
a demand before implementing.

> - as already commented  api2/session is not really Restful. I haven't read
> the details about why it's currently done this way, but wouldn't simple HTTP
> basic authentication work, at least if user and password are given?

Agreed that sessions aren't RESTful. On the other hand, there are some
backend optimizations that apparently benefit from sessions. It still
exists for two reasons:

1. It works currently and I'd like to keep the API working as I make
changes to it. I don't currently have another working method of
managing logins.
2. I don't yet understand the backend optimizations that this allows
and I'd like to understand that before ripping it out. Specifically,
the creation of a session currently starts up an Actor that I think is
used elsewhere in the API. I'm looking into this.

With regards to basic authentication, I don't want to allow the use of
basic authentication for the API. In general, token-based
authentication is a good idea for APIs as it makes explicit the
delegation that is going on when you allow an API client to act on
your behalf. There are things I would like to change about the way the
API uses the token to do authentication (either require HTTPS or don't
send the token in the clear, for starters), but I'm waiting on that
for the moment.

> - "Streaming" as far as I understand is about "push" functionality.
> I would not go as far as to say "polling is bad". Since twitter is not real
> time ESME would maybe not need to be real time as well. This means one could
> poll infrequently (every few minutes?) using conditional GET's, which would
> not be that inefficient. Comet has the disadvantage that without special
> server support (available in Jetty, but not in Netweaver CE for example)  it
> causes a lot of threads to be created on the server, which is also not
> really efficient.
> IMHO defining a pagination protocol would be more important.

Agreed, sort of. I'm defining API methods that will allow for pulling
down a list of messages or a specific message however, I agree with
David that we should provide better ways to access messages. Comet is
not currently turned on in the api2/ endpoint, but I was planning to
make it an option, though the server issue you bring up gives me
pause. What I'd really like to do is expose each message queue to
clients using AMQP (though what version I don't know).

I don't really know what the best approach is here, so I'm looking for input.

> In general I think it makes sense to define the URL pattern, but one has
> always to be careful to not fall into the trap and forget about HATEOS (
> http://blogs.sun.com/craigmcc/entry/why_hateoas).
> With proper HATEOS support, the exact URL's would be less important, and
> evolving the API  would be easier.

I've been keeping an eye on the SunCloud API development via Tim
Bray's blog but hadn't seen this angle. I'll have to think about it.
How are you thinking that this could be applied to the ESME API?

Again, thank you for your feedback. I'll start incorporating it into the design!

Ethan

Re: Patch submitted, working towards API design

[Markus Kohler <ma...@gmail.com>]

Hi Ethan,
Here come a few comments to
http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design

- it's not 100% clear to me what you mean with
api2/user/tracks
Do you mean api2/{USERID}/tracks where {USERID} is replaced by a valid
USERID?
or do you mean that depending on the authenticated user one would get back
different results?
I would not recommend to use the second approach, because that means you
cannot bookmark the URLs anymore.

-Regarding PUT and DELETE not available on all clients one option would be
to use google's approach in gdata and put an additional header into a POST
request:

X-HTTP-Method-Override: PUT



- as already commented  api2/session is not really Restful. I haven't read
the details about why it's currently done this way, but wouldn't simple HTTP
basic authentication work, at least if user and password are given?


- "Streaming" as far as I understand is about "push" functionality.
I would not go as far as to say "polling is bad". Since twitter is not real
time ESME would maybe not need to be real time as well. This means one could
poll infrequently (every few minutes?) using conditional GET's, which would
not be that inefficient. Comet has the disadvantage that without special
server support (available in Jetty, but not in Netweaver CE for example)  it
causes a lot of threads to be created on the server, which is also not
really efficient.
IMHO defining a pagination protocol would be more important.

In general I think it makes sense to define the URL pattern, but one has
always to be careful to not fall into the trap and forget about HATEOS (
http://blogs.sun.com/craigmcc/entry/why_hateoas).
With proper HATEOS support, the exact URL's would be less important, and
evolving the API  would be easier.
Since the response format is not yet specified, I think there's no issue
yet.

<http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design>Regards,
Markus

On Tue, Oct 27, 2009 at 5:17 PM, Ethan Jewett <es...@gmail.com> wrote:

> This is working on Stax!
>
> On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com>
> wrote:
> > The new api is deployed on stax.
> >
> > D.
> >
> > On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com>
> wrote:
> >> I don't see any probs with putting this in the trunk. The existing
> >> rest api remains unchanged.
> >>
> >> D.
> >>
> >> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com>
> wrote:
> >>> Hi all,
> >>>
> >>> This has been cooking for a little while (I think it was about 6 month
> >>> ago that I mentioned I should put my money/code where my mouth was),
> >>> but I finally submitted a patch for the ESME-14 Jira issue
> >>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
> >>> new branch called "new_api". The sole purpose of this branch is to
> >>> create a new API endpoint /api2/ where we can start converting the API
> >>> to a format that aligns with discussions occurring on the wiki
> >>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
> >>>
> >>> The current diff has a working API at /api2/ that has pretty much the
> >>> same methods as the old one (I just copied them after all), but
> >>> arranged in the resource/object hierarchy outlined on the wiki. I've
> >>> also moved some parameters out of the query (part of the URL after the
> >>> ?) and into the path of the URL, when these parameters actually refer
> >>> to resources/objects.
> >>>
> >>> The diff should actually exist nicely alongside the existing API and
> >>> functionality, but I figured the easiest and safest way to get it in
> >>> initial was in a branch.
> >>>
> >>> Here's what I'm looking at doing next, not necessarily in this order:
> >>>
> >>> 1. Create new handlers to fill out the object/resource hierarchy
> >>> 2. Fix up the existing code in the API2.scala file so that it all
> >>> behaves the same way
> >>> 3. Work on aligning and documenting the naming of parameters that API
> >>> handlers expect
> >>> 4. Work on accepting XML or JSON representations of new or changed
> >>> resources, rather than using query parameters
> >>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
> >>> pointers on how to make an HTTP request from Scala/Lift would be
> >>> extremely helpful, as I think this is level at which I need to test
> >>> the API.)
> >>>
> >>> See the wiki page for longer-term discussions and approaches.
> >>>
> >>> Any comments or questions? I've got to note that this is the first
> >>> Scala I've ever written (though truly, it's mostly copied and adjusted
> >>> at this point) outside of examples from books, so please please please
> >>> tell me how I should be doing things differently.
> >>>
> >>> Thanks,
> >>> Ethan
> >>>
> >>
> >
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

Committed and deployed on stax

On Sat, Oct 31, 2009 at 12:12 AM, Ethan Jewett <es...@gmail.com> wrote:
> Dick,
>
> There is a new patch on the Jira item. The current check-in won't run
> (as far as I can tell) because the boot.scala doesn't include the
> API2.
>
> Will be updating the wiki shortly with proposed methods to add and
> more documentation of how it is currently working.
>
> Ethan
>
> On Thu, Oct 29, 2009 at 9:58 AM, Richard Hirsch <hi...@gmail.com> wrote:
>> Looking forward to seing the next version. I'll do a deplyoment after
>> I see the code updated in Jira.
>>
>> D.
>>
>> On Thu, Oct 29, 2009 at 3:51 PM, Ethan Jewett <es...@gmail.com> wrote:
>>> I'm hoping to be able to expand on that this weekend a bit. I'm
>>> updating http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design
>>> to keep track a bit, and if people want more resources in the API they
>>> should add them there (and discuss on this list).
>>>
>>> Currently Tags have been left out, Searches are an item that we might
>>> want to add, and there are no stream-based interfaces to message
>>> queues, which I now agree are very important. Lots of work to do there
>>> :-) I'll try to get this stuff written up and into Jira over the
>>> weekend.
>>>
>>> Ethan
>>>
>>> On Thu, Oct 29, 2009 at 10:19 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>> I've committed the changes - what is missing in the API?
>>>>
>>>> D.
>>>>
>>>> On Tue, Oct 27, 2009 at 5:17 PM, Ethan Jewett <es...@gmail.com> wrote:
>>>>> This is working on Stax!
>>>>>
>>>>> On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>>>> The new api is deployed on stax.
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>>>>> I don't see any probs with putting this in the trunk. The existing
>>>>>>> rest api remains unchanged.
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>>>>> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> This has been cooking for a little while (I think it was about 6 month
>>>>>>>> ago that I mentioned I should put my money/code where my mouth was),
>>>>>>>> but I finally submitted a patch for the ESME-14 Jira issue
>>>>>>>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>>>>>>>> new branch called "new_api". The sole purpose of this branch is to
>>>>>>>> create a new API endpoint /api2/ where we can start converting the API
>>>>>>>> to a format that aligns with discussions occurring on the wiki
>>>>>>>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>>>>>>>
>>>>>>>> The current diff has a working API at /api2/ that has pretty much the
>>>>>>>> same methods as the old one (I just copied them after all), but
>>>>>>>> arranged in the resource/object hierarchy outlined on the wiki. I've
>>>>>>>> also moved some parameters out of the query (part of the URL after the
>>>>>>>> ?) and into the path of the URL, when these parameters actually refer
>>>>>>>> to resources/objects.
>>>>>>>>
>>>>>>>> The diff should actually exist nicely alongside the existing API and
>>>>>>>> functionality, but I figured the easiest and safest way to get it in
>>>>>>>> initial was in a branch.
>>>>>>>>
>>>>>>>> Here's what I'm looking at doing next, not necessarily in this order:
>>>>>>>>
>>>>>>>> 1. Create new handlers to fill out the object/resource hierarchy
>>>>>>>> 2. Fix up the existing code in the API2.scala file so that it all
>>>>>>>> behaves the same way
>>>>>>>> 3. Work on aligning and documenting the naming of parameters that API
>>>>>>>> handlers expect
>>>>>>>> 4. Work on accepting XML or JSON representations of new or changed
>>>>>>>> resources, rather than using query parameters
>>>>>>>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>>>>>>>> pointers on how to make an HTTP request from Scala/Lift would be
>>>>>>>> extremely helpful, as I think this is level at which I need to test
>>>>>>>> the API.)
>>>>>>>>
>>>>>>>> See the wiki page for longer-term discussions and approaches.
>>>>>>>>
>>>>>>>> Any comments or questions? I've got to note that this is the first
>>>>>>>> Scala I've ever written (though truly, it's mostly copied and adjusted
>>>>>>>> at this point) outside of examples from books, so please please please
>>>>>>>> tell me how I should be doing things differently.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ethan
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

Dick,

There is a new patch on the Jira item. The current check-in won't run
(as far as I can tell) because the boot.scala doesn't include the
API2.

Will be updating the wiki shortly with proposed methods to add and
more documentation of how it is currently working.

Ethan

On Thu, Oct 29, 2009 at 9:58 AM, Richard Hirsch <hi...@gmail.com> wrote:
> Looking forward to seing the next version. I'll do a deplyoment after
> I see the code updated in Jira.
>
> D.
>
> On Thu, Oct 29, 2009 at 3:51 PM, Ethan Jewett <es...@gmail.com> wrote:
>> I'm hoping to be able to expand on that this weekend a bit. I'm
>> updating http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design
>> to keep track a bit, and if people want more resources in the API they
>> should add them there (and discuss on this list).
>>
>> Currently Tags have been left out, Searches are an item that we might
>> want to add, and there are no stream-based interfaces to message
>> queues, which I now agree are very important. Lots of work to do there
>> :-) I'll try to get this stuff written up and into Jira over the
>> weekend.
>>
>> Ethan
>>
>> On Thu, Oct 29, 2009 at 10:19 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>> I've committed the changes - what is missing in the API?
>>>
>>> D.
>>>
>>> On Tue, Oct 27, 2009 at 5:17 PM, Ethan Jewett <es...@gmail.com> wrote:
>>>> This is working on Stax!
>>>>
>>>> On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>>> The new api is deployed on stax.
>>>>>
>>>>> D.
>>>>>
>>>>> On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>>>> I don't see any probs with putting this in the trunk. The existing
>>>>>> rest api remains unchanged.
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> This has been cooking for a little while (I think it was about 6 month
>>>>>>> ago that I mentioned I should put my money/code where my mouth was),
>>>>>>> but I finally submitted a patch for the ESME-14 Jira issue
>>>>>>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>>>>>>> new branch called "new_api". The sole purpose of this branch is to
>>>>>>> create a new API endpoint /api2/ where we can start converting the API
>>>>>>> to a format that aligns with discussions occurring on the wiki
>>>>>>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>>>>>>
>>>>>>> The current diff has a working API at /api2/ that has pretty much the
>>>>>>> same methods as the old one (I just copied them after all), but
>>>>>>> arranged in the resource/object hierarchy outlined on the wiki. I've
>>>>>>> also moved some parameters out of the query (part of the URL after the
>>>>>>> ?) and into the path of the URL, when these parameters actually refer
>>>>>>> to resources/objects.
>>>>>>>
>>>>>>> The diff should actually exist nicely alongside the existing API and
>>>>>>> functionality, but I figured the easiest and safest way to get it in
>>>>>>> initial was in a branch.
>>>>>>>
>>>>>>> Here's what I'm looking at doing next, not necessarily in this order:
>>>>>>>
>>>>>>> 1. Create new handlers to fill out the object/resource hierarchy
>>>>>>> 2. Fix up the existing code in the API2.scala file so that it all
>>>>>>> behaves the same way
>>>>>>> 3. Work on aligning and documenting the naming of parameters that API
>>>>>>> handlers expect
>>>>>>> 4. Work on accepting XML or JSON representations of new or changed
>>>>>>> resources, rather than using query parameters
>>>>>>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>>>>>>> pointers on how to make an HTTP request from Scala/Lift would be
>>>>>>> extremely helpful, as I think this is level at which I need to test
>>>>>>> the API.)
>>>>>>>
>>>>>>> See the wiki page for longer-term discussions and approaches.
>>>>>>>
>>>>>>> Any comments or questions? I've got to note that this is the first
>>>>>>> Scala I've ever written (though truly, it's mostly copied and adjusted
>>>>>>> at this point) outside of examples from books, so please please please
>>>>>>> tell me how I should be doing things differently.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ethan
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

Looking forward to seing the next version. I'll do a deplyoment after
I see the code updated in Jira.

D.

On Thu, Oct 29, 2009 at 3:51 PM, Ethan Jewett <es...@gmail.com> wrote:
> I'm hoping to be able to expand on that this weekend a bit. I'm
> updating http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design
> to keep track a bit, and if people want more resources in the API they
> should add them there (and discuss on this list).
>
> Currently Tags have been left out, Searches are an item that we might
> want to add, and there are no stream-based interfaces to message
> queues, which I now agree are very important. Lots of work to do there
> :-) I'll try to get this stuff written up and into Jira over the
> weekend.
>
> Ethan
>
> On Thu, Oct 29, 2009 at 10:19 AM, Richard Hirsch <hi...@gmail.com> wrote:
>> I've committed the changes - what is missing in the API?
>>
>> D.
>>
>> On Tue, Oct 27, 2009 at 5:17 PM, Ethan Jewett <es...@gmail.com> wrote:
>>> This is working on Stax!
>>>
>>> On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>> The new api is deployed on stax.
>>>>
>>>> D.
>>>>
>>>> On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>>> I don't see any probs with putting this in the trunk. The existing
>>>>> rest api remains unchanged.
>>>>>
>>>>> D.
>>>>>
>>>>> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> This has been cooking for a little while (I think it was about 6 month
>>>>>> ago that I mentioned I should put my money/code where my mouth was),
>>>>>> but I finally submitted a patch for the ESME-14 Jira issue
>>>>>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>>>>>> new branch called "new_api". The sole purpose of this branch is to
>>>>>> create a new API endpoint /api2/ where we can start converting the API
>>>>>> to a format that aligns with discussions occurring on the wiki
>>>>>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>>>>>
>>>>>> The current diff has a working API at /api2/ that has pretty much the
>>>>>> same methods as the old one (I just copied them after all), but
>>>>>> arranged in the resource/object hierarchy outlined on the wiki. I've
>>>>>> also moved some parameters out of the query (part of the URL after the
>>>>>> ?) and into the path of the URL, when these parameters actually refer
>>>>>> to resources/objects.
>>>>>>
>>>>>> The diff should actually exist nicely alongside the existing API and
>>>>>> functionality, but I figured the easiest and safest way to get it in
>>>>>> initial was in a branch.
>>>>>>
>>>>>> Here's what I'm looking at doing next, not necessarily in this order:
>>>>>>
>>>>>> 1. Create new handlers to fill out the object/resource hierarchy
>>>>>> 2. Fix up the existing code in the API2.scala file so that it all
>>>>>> behaves the same way
>>>>>> 3. Work on aligning and documenting the naming of parameters that API
>>>>>> handlers expect
>>>>>> 4. Work on accepting XML or JSON representations of new or changed
>>>>>> resources, rather than using query parameters
>>>>>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>>>>>> pointers on how to make an HTTP request from Scala/Lift would be
>>>>>> extremely helpful, as I think this is level at which I need to test
>>>>>> the API.)
>>>>>>
>>>>>> See the wiki page for longer-term discussions and approaches.
>>>>>>
>>>>>> Any comments or questions? I've got to note that this is the first
>>>>>> Scala I've ever written (though truly, it's mostly copied and adjusted
>>>>>> at this point) outside of examples from books, so please please please
>>>>>> tell me how I should be doing things differently.
>>>>>>
>>>>>> Thanks,
>>>>>> Ethan
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

I'm hoping to be able to expand on that this weekend a bit. I'm
updating http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design
to keep track a bit, and if people want more resources in the API they
should add them there (and discuss on this list).

Currently Tags have been left out, Searches are an item that we might
want to add, and there are no stream-based interfaces to message
queues, which I now agree are very important. Lots of work to do there
:-) I'll try to get this stuff written up and into Jira over the
weekend.

Ethan

On Thu, Oct 29, 2009 at 10:19 AM, Richard Hirsch <hi...@gmail.com> wrote:
> I've committed the changes - what is missing in the API?
>
> D.
>
> On Tue, Oct 27, 2009 at 5:17 PM, Ethan Jewett <es...@gmail.com> wrote:
>> This is working on Stax!
>>
>> On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>> The new api is deployed on stax.
>>>
>>> D.
>>>
>>> On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>>> I don't see any probs with putting this in the trunk. The existing
>>>> rest api remains unchanged.
>>>>
>>>> D.
>>>>
>>>> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>>>>> Hi all,
>>>>>
>>>>> This has been cooking for a little while (I think it was about 6 month
>>>>> ago that I mentioned I should put my money/code where my mouth was),
>>>>> but I finally submitted a patch for the ESME-14 Jira issue
>>>>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>>>>> new branch called "new_api". The sole purpose of this branch is to
>>>>> create a new API endpoint /api2/ where we can start converting the API
>>>>> to a format that aligns with discussions occurring on the wiki
>>>>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>>>>
>>>>> The current diff has a working API at /api2/ that has pretty much the
>>>>> same methods as the old one (I just copied them after all), but
>>>>> arranged in the resource/object hierarchy outlined on the wiki. I've
>>>>> also moved some parameters out of the query (part of the URL after the
>>>>> ?) and into the path of the URL, when these parameters actually refer
>>>>> to resources/objects.
>>>>>
>>>>> The diff should actually exist nicely alongside the existing API and
>>>>> functionality, but I figured the easiest and safest way to get it in
>>>>> initial was in a branch.
>>>>>
>>>>> Here's what I'm looking at doing next, not necessarily in this order:
>>>>>
>>>>> 1. Create new handlers to fill out the object/resource hierarchy
>>>>> 2. Fix up the existing code in the API2.scala file so that it all
>>>>> behaves the same way
>>>>> 3. Work on aligning and documenting the naming of parameters that API
>>>>> handlers expect
>>>>> 4. Work on accepting XML or JSON representations of new or changed
>>>>> resources, rather than using query parameters
>>>>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>>>>> pointers on how to make an HTTP request from Scala/Lift would be
>>>>> extremely helpful, as I think this is level at which I need to test
>>>>> the API.)
>>>>>
>>>>> See the wiki page for longer-term discussions and approaches.
>>>>>
>>>>> Any comments or questions? I've got to note that this is the first
>>>>> Scala I've ever written (though truly, it's mostly copied and adjusted
>>>>> at this point) outside of examples from books, so please please please
>>>>> tell me how I should be doing things differently.
>>>>>
>>>>> Thanks,
>>>>> Ethan
>>>>>
>>>>
>>>
>>
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

I've committed the changes - what is missing in the API?

D.

On Tue, Oct 27, 2009 at 5:17 PM, Ethan Jewett <es...@gmail.com> wrote:
> This is working on Stax!
>
> On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com> wrote:
>> The new api is deployed on stax.
>>
>> D.
>>
>> On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
>>> I don't see any probs with putting this in the trunk. The existing
>>> rest api remains unchanged.
>>>
>>> D.
>>>
>>> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>>>> Hi all,
>>>>
>>>> This has been cooking for a little while (I think it was about 6 month
>>>> ago that I mentioned I should put my money/code where my mouth was),
>>>> but I finally submitted a patch for the ESME-14 Jira issue
>>>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>>>> new branch called "new_api". The sole purpose of this branch is to
>>>> create a new API endpoint /api2/ where we can start converting the API
>>>> to a format that aligns with discussions occurring on the wiki
>>>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>>>
>>>> The current diff has a working API at /api2/ that has pretty much the
>>>> same methods as the old one (I just copied them after all), but
>>>> arranged in the resource/object hierarchy outlined on the wiki. I've
>>>> also moved some parameters out of the query (part of the URL after the
>>>> ?) and into the path of the URL, when these parameters actually refer
>>>> to resources/objects.
>>>>
>>>> The diff should actually exist nicely alongside the existing API and
>>>> functionality, but I figured the easiest and safest way to get it in
>>>> initial was in a branch.
>>>>
>>>> Here's what I'm looking at doing next, not necessarily in this order:
>>>>
>>>> 1. Create new handlers to fill out the object/resource hierarchy
>>>> 2. Fix up the existing code in the API2.scala file so that it all
>>>> behaves the same way
>>>> 3. Work on aligning and documenting the naming of parameters that API
>>>> handlers expect
>>>> 4. Work on accepting XML or JSON representations of new or changed
>>>> resources, rather than using query parameters
>>>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>>>> pointers on how to make an HTTP request from Scala/Lift would be
>>>> extremely helpful, as I think this is level at which I need to test
>>>> the API.)
>>>>
>>>> See the wiki page for longer-term discussions and approaches.
>>>>
>>>> Any comments or questions? I've got to note that this is the first
>>>> Scala I've ever written (though truly, it's mostly copied and adjusted
>>>> at this point) outside of examples from books, so please please please
>>>> tell me how I should be doing things differently.
>>>>
>>>> Thanks,
>>>> Ethan
>>>>
>>>
>>
>

Re: Patch submitted, working towards API design

[Ethan Jewett <es...@gmail.com>]

This is working on Stax!

On Tue, Oct 27, 2009 at 3:41 AM, Richard Hirsch <hi...@gmail.com> wrote:
> The new api is deployed on stax.
>
> D.
>
> On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
>> I don't see any probs with putting this in the trunk. The existing
>> rest api remains unchanged.
>>
>> D.
>>
>> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>>> Hi all,
>>>
>>> This has been cooking for a little while (I think it was about 6 month
>>> ago that I mentioned I should put my money/code where my mouth was),
>>> but I finally submitted a patch for the ESME-14 Jira issue
>>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>>> new branch called "new_api". The sole purpose of this branch is to
>>> create a new API endpoint /api2/ where we can start converting the API
>>> to a format that aligns with discussions occurring on the wiki
>>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>>
>>> The current diff has a working API at /api2/ that has pretty much the
>>> same methods as the old one (I just copied them after all), but
>>> arranged in the resource/object hierarchy outlined on the wiki. I've
>>> also moved some parameters out of the query (part of the URL after the
>>> ?) and into the path of the URL, when these parameters actually refer
>>> to resources/objects.
>>>
>>> The diff should actually exist nicely alongside the existing API and
>>> functionality, but I figured the easiest and safest way to get it in
>>> initial was in a branch.
>>>
>>> Here's what I'm looking at doing next, not necessarily in this order:
>>>
>>> 1. Create new handlers to fill out the object/resource hierarchy
>>> 2. Fix up the existing code in the API2.scala file so that it all
>>> behaves the same way
>>> 3. Work on aligning and documenting the naming of parameters that API
>>> handlers expect
>>> 4. Work on accepting XML or JSON representations of new or changed
>>> resources, rather than using query parameters
>>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>>> pointers on how to make an HTTP request from Scala/Lift would be
>>> extremely helpful, as I think this is level at which I need to test
>>> the API.)
>>>
>>> See the wiki page for longer-term discussions and approaches.
>>>
>>> Any comments or questions? I've got to note that this is the first
>>> Scala I've ever written (though truly, it's mostly copied and adjusted
>>> at this point) outside of examples from books, so please please please
>>> tell me how I should be doing things differently.
>>>
>>> Thanks,
>>> Ethan
>>>
>>
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

The new api is deployed on stax.

D.

On Mon, Oct 26, 2009 at 8:09 AM, Richard Hirsch <hi...@gmail.com> wrote:
> I don't see any probs with putting this in the trunk. The existing
> rest api remains unchanged.
>
> D.
>
> On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
>> Hi all,
>>
>> This has been cooking for a little while (I think it was about 6 month
>> ago that I mentioned I should put my money/code where my mouth was),
>> but I finally submitted a patch for the ESME-14 Jira issue
>> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
>> new branch called "new_api". The sole purpose of this branch is to
>> create a new API endpoint /api2/ where we can start converting the API
>> to a format that aligns with discussions occurring on the wiki
>> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>>
>> The current diff has a working API at /api2/ that has pretty much the
>> same methods as the old one (I just copied them after all), but
>> arranged in the resource/object hierarchy outlined on the wiki. I've
>> also moved some parameters out of the query (part of the URL after the
>> ?) and into the path of the URL, when these parameters actually refer
>> to resources/objects.
>>
>> The diff should actually exist nicely alongside the existing API and
>> functionality, but I figured the easiest and safest way to get it in
>> initial was in a branch.
>>
>> Here's what I'm looking at doing next, not necessarily in this order:
>>
>> 1. Create new handlers to fill out the object/resource hierarchy
>> 2. Fix up the existing code in the API2.scala file so that it all
>> behaves the same way
>> 3. Work on aligning and documenting the naming of parameters that API
>> handlers expect
>> 4. Work on accepting XML or JSON representations of new or changed
>> resources, rather than using query parameters
>> 5. Write tests/specs (I'm working on figuring out how to do this. Any
>> pointers on how to make an HTTP request from Scala/Lift would be
>> extremely helpful, as I think this is level at which I need to test
>> the API.)
>>
>> See the wiki page for longer-term discussions and approaches.
>>
>> Any comments or questions? I've got to note that this is the first
>> Scala I've ever written (though truly, it's mostly copied and adjusted
>> at this point) outside of examples from books, so please please please
>> tell me how I should be doing things differently.
>>
>> Thanks,
>> Ethan
>>
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

I don't see any probs with putting this in the trunk. The existing
rest api remains unchanged.

D.

On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
> Hi all,
>
> This has been cooking for a little while (I think it was about 6 month
> ago that I mentioned I should put my money/code where my mouth was),
> but I finally submitted a patch for the ESME-14 Jira issue
> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
> new branch called "new_api". The sole purpose of this branch is to
> create a new API endpoint /api2/ where we can start converting the API
> to a format that aligns with discussions occurring on the wiki
> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>
> The current diff has a working API at /api2/ that has pretty much the
> same methods as the old one (I just copied them after all), but
> arranged in the resource/object hierarchy outlined on the wiki. I've
> also moved some parameters out of the query (part of the URL after the
> ?) and into the path of the URL, when these parameters actually refer
> to resources/objects.
>
> The diff should actually exist nicely alongside the existing API and
> functionality, but I figured the easiest and safest way to get it in
> initial was in a branch.
>
> Here's what I'm looking at doing next, not necessarily in this order:
>
> 1. Create new handlers to fill out the object/resource hierarchy
> 2. Fix up the existing code in the API2.scala file so that it all
> behaves the same way
> 3. Work on aligning and documenting the naming of parameters that API
> handlers expect
> 4. Work on accepting XML or JSON representations of new or changed
> resources, rather than using query parameters
> 5. Write tests/specs (I'm working on figuring out how to do this. Any
> pointers on how to make an HTTP request from Scala/Lift would be
> extremely helpful, as I think this is level at which I need to test
> the API.)
>
> See the wiki page for longer-term discussions and approaches.
>
> Any comments or questions? I've got to note that this is the first
> Scala I've ever written (though truly, it's mostly copied and adjusted
> at this point) outside of examples from books, so please please please
> tell me how I should be doing things differently.
>
> Thanks,
> Ethan
>

Re: Patch submitted, working towards API design

[Richard Hirsch <hi...@gmail.com>]

Sounds good. I'll do a deplyoment based on the patch as as soon as I
can. I leave for the TechED today, so things might be a little hectic
the next few days.

D.

On Mon, Oct 26, 2009 at 1:28 AM, Ethan Jewett <es...@gmail.com> wrote:
> Hi all,
>
> This has been cooking for a little while (I think it was about 6 month
> ago that I mentioned I should put my money/code where my mouth was),
> but I finally submitted a patch for the ESME-14 Jira issue
> (https://issues.apache.org/jira/browse/ESME-14). This patch creates a
> new branch called "new_api". The sole purpose of this branch is to
> create a new API endpoint /api2/ where we can start converting the API
> to a format that aligns with discussions occurring on the wiki
> (http://cwiki.apache.org/confluence/display/ESME/API+2.0+-+Design).
>
> The current diff has a working API at /api2/ that has pretty much the
> same methods as the old one (I just copied them after all), but
> arranged in the resource/object hierarchy outlined on the wiki. I've
> also moved some parameters out of the query (part of the URL after the
> ?) and into the path of the URL, when these parameters actually refer
> to resources/objects.
>
> The diff should actually exist nicely alongside the existing API and
> functionality, but I figured the easiest and safest way to get it in
> initial was in a branch.
>
> Here's what I'm looking at doing next, not necessarily in this order:
>
> 1. Create new handlers to fill out the object/resource hierarchy
> 2. Fix up the existing code in the API2.scala file so that it all
> behaves the same way
> 3. Work on aligning and documenting the naming of parameters that API
> handlers expect
> 4. Work on accepting XML or JSON representations of new or changed
> resources, rather than using query parameters
> 5. Write tests/specs (I'm working on figuring out how to do this. Any
> pointers on how to make an HTTP request from Scala/Lift would be
> extremely helpful, as I think this is level at which I need to test
> the API.)
>
> See the wiki page for longer-term discussions and approaches.
>
> Any comments or questions? I've got to note that this is the first
> Scala I've ever written (though truly, it's mostly copied and adjusted
> at this point) outside of examples from books, so please please please
> tell me how I should be doing things differently.
>
> Thanks,
> Ethan
>