You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rave.apache.org by Matt Franklin <m....@gmail.com> on 2013/10/06 22:56:32 UTC
Re: Rave Data Structures
On Tue, Sep 24, 2013 at 1:02 PM, Chris Geer <ch...@cxtsoftware.com> wrote:
> On Tue, Sep 24, 2013 at 9:54 AM, Michael Jett <mi...@gmail.com> wrote:
>
> > On Mon, Sep 23, 2013 at 7:28 PM, Chris Geer <ch...@cxtsoftware.com>
> wrote:
> >
> > > Some additional comments below:
> > >
> > > On Sun, Sep 22, 2013 at 7:01 PM, Erin Noe-Payne <
> > erin.noe.payne@gmail.com
> > > >wrote:
> > >
> > > > Hey all, sorry I've been absent from this thread. I've caught up now
> > > > and have a few thoughts:
> > > >
> > > > - As discussed before, the REST api should have CRUD endpoints that
> > > > support all data retrieval and manipulation for any case that we wish
> > > > for RAVE to allow. These endpoints should not be optimized for
> > > > specific client application / library needs.
> > > > - In addition we have some number of endpoints (currently one) that
> > > > optimize for specific client needs. In this case we are talking about
> > > > the pages for render endpoint, which serves 3 purposes:
> > > > 1) Filter to the set of pages by context and identifier
> > > > 2) Aggregate nested data for one read operation that gives all data
> > > > necessary to render
> > > > 3) Attach special properties to regionWidgets needed to render the
> > > > iframes (rpc tokens)
> > > > - The data structure we retrieve from the render endpoint should not
> > > > be different from the models sent / consumed by the CRUD endpoints.
> > > >
> > >
> > > So using your example above....the regionWidget CRUD model should have
> > rpc
> > > token?
> > >
> > >
> > > > The render endpoint is read-only, so any data manipulation that
> > > > follows will be interacting with the CRUD endpoints. This is not
> > > > specific to angular - any client technology will be consuming the
> > > > render endpoint and feeding back that data in the case of updates.
> > > > Again, render endpoint is only filtering, aggregating nested data,
> > > > attaching render properties to regionWidget. Otherwise I don't see a
> > > > reason the data models should differ.
> > > >
> > > > -A regionwidget should probably just have a mutable title property.
> At
> > > > time of creation the title can be copied from the widget it is an
> > > > instance of, but afterwards it can be changed by the user. F.E if I
> > > > have 3 instances of a map widget or a weather widget or something on
> > > > my page, I may want to title them differently anyway.
> > > >
> > >
> > > I support this +1 as long as it's mutable.
> > >
> > > >
> > > > - I don't really know the best way to deal with relationships. My
> > > > instinct is that resources that have a one-to-X relationship should
> > > > simply have the id of that relationship as a property. For example,
> > > > each region has a pageId property. Each regionWidget has a pageId and
> > > > regionId property. Then these relationships are atomically mutable
> via
> > > > a PUT request. Many-to-many relationships should be modeled via their
> > > > own resource. So for example user-to-users friend should correspond
> to
> > > > a friends resource that can be POSTed and DELETEd.
> > > >
> > >
> > > My opinion is our data model is bad on the server. A RegionWidget
> > shouldn't
> > > have a RegionID or PageID because they really shouldn't be mutable. I
> > > realize this is a backend problem, not a REST problem exactly. I'm just
> > > arguing we don't extend that to the REST interface so we can fix the
> > > backend problem later. I don't like a situation where you can have a
> url
> > > and data that can contradict.
> > >
> >
> > +1 - I think that this is a great point Chris. What planning or
> > coordination would it take to fix this backend problem?
> >
>
> I don't think fixing the backend problem is our primary concern as we can
> work around it. My recommendation would be to design the REST API in such a
> way as it makes the most sense, then we can fix the backend.
>
> >
> >
> > >
> > > > - Angular doesn't really care about this stuff. We have started
> > > > implementing according to what exists currently. If the api changes,
> > > > then it will create some refactoring on the client. That's fine -
> > > > $resource is just json objects anyway, so it can handle it.
> However...
> > > > - We really just need consistency. My impression so far is that we
> > > > don't really have any practical expertise on building a robust REST
> > > > api around a complicated data set. Short of that expertise showing up
> > > > on the list, we really just need to rally around one strategy and
> > > > implement it consistently.
> > > >
> > >
> > > Well, that's not entirely true ;) At the end of the day, this data
> model
> > > really isn't all that complex (the rendering part). Page contains 0-n
> > > Regions which contain 0-n Region Widgets. There is no reason why a
> > > RegionWidget needs a reference to the page it's on directly, that
> should
> > be
> > > derived by which resource it's retrieved from. It's that way on the
> > backend
> > > because our backed just mirrors the SQL table and it's foreign keys. If
> > we
> > > had started this project with a NoSQL database, we never would have had
> > > those attributes on those objects (probably wouldn't have used JPA
> > either).
> > >
> > > Using Mongo as an example, we never would have done this for a page...
> > >
> > > {
> > > id: 123,
> > > regions: [
> > > {
> > > id: 332,
> > > page_id: 123,
> > > region_widgets: [
> > > {
> > > id: 454,
> > > region_id: 332
> > > }
> > > ]
> > > }
> > > ]
> > > }
> > >
> > > We would have stored page as a single document and not broken Regions
> and
> > > RegionWidgets out into separate top level objects.
> > >
> > >
> > That makes sense. The REST interface would give us an opportunity to
> > consolidate and hide the relational keys, presenting objects in a
> "natural"
> > state. (i.e. pages, users) Rather than breaking it down into proprietary
> > sub-object jargon. (i.e. RegionWidgets...)
> >
>
> Bingo!! Now we are on the same page.
>
I think this all makes sense. The question now is if anyone has captured
it in Jira or started implementing it. I was looking at the Page rest
model object today in trunk and it is still the deep one we created to
support the full "rendered" object.
>
> >
> >
> > > Chris
> > >
> > >
> > > >
> > > > On Wed, Sep 18, 2013 at 1:29 PM, Chris Geer <ch...@cxtsoftware.com>
> > > wrote:
> > > > > Dan,
> > > > >
> > > > > My proposal is that on the server side, we do not add the extra
> > > > attributes
> > > > > to the REST model. Instead we have the render endpoint that will
> > return
> > > > the
> > > > > full object tree with the extra attributes. So on the server side
> > there
> > > > > would be a different object like you said i.e.
> RenderedRegionWidget.
> > An
> > > > > alternative is to not create an objet and just build the JSON
> > directly
> > > > for
> > > > > the render endpoint.
> > > > >
> > > > > I'm also ok with the server ignoring unknown attributes on a
> > > save/update.
> > > > >
> > > > > Chris
> > > > >
> > > > >
> > > > > On Wed, Sep 18, 2013 at 3:56 AM, Dan Gornstein <da...@gornstein.com>
> > > > wrote:
> > > > >
> > > > >> Chris,
> > > > >>
> > > > >> I think I was a bit confused before and might have been inaccurate
> > in
> > > > what
> > > > >> I was trying to say. I have been thinking about it a bit more had
> a
> > > few
> > > > >> more questions for you.
> > > > >>
> > > > >> The endpoint which returns information for rendering a page is a
> > > nested
> > > > >> object and I agree should be read only, meaning you cannot post to
> > the
> > > > same
> > > > >> endpoint with the nested data model to save your object. In the
> > > angular
> > > > >> branch right now, when we hit this endpoint we break down the
> > response
> > > > into
> > > > >> their individual angular resource objects (page, regions,
> > > regionWidgets)
> > > > >> which point to the CRUD interfaces.
> > > > >>
> > > > >> The way the pages for render endpoint works right now is to use
> > > > pageService
> > > > >> to get the canonical model of a page and transform it into the
> REST
> > > > model
> > > > >> and send it through the DefaultRenderService to prepare the
> > > > regionWidgets.
> > > > >> This means it ends up returning the REST model of RegionWidgets in
> > the
> > > > >> pages for render endpoint. So if you wanted to add the pageId and
> > > title,
> > > > >> they would have to be properties on the RegionWidget REST model.
> Are
> > > you
> > > > >> proposing that we make this pages for render endpoint be compiled
> > with
> > > > >> different model objects other than the REST ones? Would there be
> > > > something
> > > > >> along the lines of RegionWidgetRender, which has the extra
> > properties
> > > > but
> > > > >> does not have CRUD operations?
> > > > >>
> > > > >> If this was the case and we did not add pageId and title to the
> REST
> > > > models
> > > > >> of RegionWidgets, what will happen if we hit the pages for render
> > > > endpoint
> > > > >> and in angular still broke them down into their individual
> resource
> > > > >> objects? Would you be allowed to save a RegionWidget to
> > > > >>
> > /pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id>
> > > > which
> > > > >> had title and pageId on it and the server would ignore it, or
> would
> > > you
> > > > >> expect the server to throw an error?
> > > > >>
> > > > >> Thanks,
> > > > >> Dan
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Mon, Sep 16, 2013 at 1:21 PM, Michael Jett <
> mikesjett@gmail.com>
> > > > wrote:
> > > > >>
> > > > >> > I'd like to pause this discussion for a brief moment, and advise
> > > that
> > > > we
> > > > >> > watch this APIGee video.
> > > > >> >
> > http://www.youtube.com/watch?feature=player_embedded&v=QpAhXa12xvU
> > > > >> >
> > > > >> > This is something that Erin pointed us to early on, and serves
> as
> > a
> > > > basis
> > > > >> > for a good RESTful architecture. There are too many good points
> > made
> > > > that
> > > > >> > will allow our API to scale and be intuitive for other
> developers.
> > > > >> >
> > > > >> > It's very important that we are on the same page. I believe a
> lot
> > of
> > > > >> > "format" questions will be answered by following the APIGee
> > > > guidelines.
> > > > >> >
> > > > >> > - Mike
> > > > >> >
> > > > >> >
> > > > >> > On Sun, Sep 15, 2013 at 6:30 PM, Chris Geer <
> > chris@cxtsoftware.com>
> > > > >> wrote:
> > > > >> >
> > > > >> > > Since we've been having a lot of discussions on data
> structures
> > > > lately
> > > > >> I
> > > > >> > > wanted to write down what my suggestions were. These aren't
> 100%
> > > > >> complete
> > > > >> > > examples but show the relationships
> > > > >> > >
> > > > >> > > CRUD Interfaces
> > > > >> > >
> > > > >> > > Page (/pages/<page-id>)
> > > > >> > > {
> > > > >> > > id: <id>,
> > > > >> > > owner: <owner>,
> > > > >> > > regionIds: [
> > > > >> > > <id>, <id>, <id> (region order is based on order in
> list,
> > > not
> > > > >> field
> > > > >> > > on region object)
> > > > >> > > ]
> > > > >> > > }
> > > > >> > >
> > > > >> > > Region (/pages/<page-id>/regions/<region-id>
> > > > >> > > {
> > > > >> > > id: <id>,
> > > > >> > > regionWidgetIds: [
> > > > >> > > <id>, <id>, <id> (widget order is based on order in
> list,
> > > not
> > > > >> field
> > > > >> > > on region widget object)
> > > > >> > > ]
> > > > >> > > }
> > > > >> > >
> > > > >> > > Region Widget
> > > > >> > >
> > > >
> (/pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id>
> > > > >> > > {
> > > > >> > > id: <id>,
> > > > >> > > widgetId: <widgetId>,
> > > > >> > > collapsed: <collapsed>,
> > > > >> > > ...
> > > > >> > > }
> > > > >> > >
> > > > >> > > Region Widget
> > > > >> > > Properties
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> (/pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id/properties/<propertyId>)
> > > > >> > > {
> > > > >> > > ....
> > > > >> > > }
> > > > >> > >
> > > > >> > > Widget (/widgets/<widget-id>
> > > > >> > > {
> > > > >> > > id: <id>
> > > > >> > > title: <title>
> > > > >> > > }
> > > > >> > >
> > > > >> > >
> > > > >> > > Render Interface (custom mime-type) (Read Only)
> > > > >> > >
> > > > >> > > /pages/<page-id>
> > > > >> > > {
> > > > >> > > id: <id>,
> > > > >> > > regions: [
> > > > >> > > {
> > > > >> > > id: <id>,
> > > > >> > > regionWidgets: [
> > > > >> > > {
> > > > >> > > id: <id>,
> > > > >> > > widgetId: <widgetId>,
> > > > >> > > title: <title>,
> > > > >> > > properties: [
> > > > >> > > {
> > > > >> > > key: <key>,
> > > > >> > > value: <value>
> > > > >> > > }
> > > > >> > > ]
> > > > >> > > }
> > > > >> > > ]
> > > > >> > > }
> > > > >> > > ]
> > > > >> > > }
> > > > >> > >
> > > > >> > > You should also be able to render sub elements below a page so
> > for
> > > > >> > example,
> > > > >> > >
> > > > >> > > /pages/<page-id>/regions/<region-id> with the custom mime-type
> > > would
> > > > >> > render
> > > > >> > > a single region.
> > > > >> > >
> > > > >> > >
> > > > >> > > Obviously there is still room for uncertainty in some places.
> > For
> > > > >> > example,
> > > > >> > > what happens if your have a region with three region widgets
> > then
> > > > you
> > > > >> > save
> > > > >> > > the region but only include two ids in the list? Personally, I
> > > think
> > > > >> that
> > > > >> > > should delete the missing regionWidget because that list
> denotes
> > > > >> > ordering.
> > > > >> > > The reason I don't like an "order" attribute on the sub
> objects
> > is
> > > > that
> > > > >> > > what if you save two sub objects with the same order (which
> > would
> > > > >> happen
> > > > >> > if
> > > > >> > > you ever wanted to swap two objects in order because you have
> to
> > > > update
> > > > >> > > them then save each one so they would have the same order at
> > least
> > > > >> > > momentarily on the server)?
> > > > >> > >
> > > > >> > > Anyway, my 2-cents.
> > > > >> > >
> > > > >> > > Chris
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
Re: Rave Data Structures
Posted by Chris Geer <ch...@cxtsoftware.com>.
Better late than never. I started to document my thoughts on the wiki.
https://wiki.apache.org/rave/RESTAPI
Chris
On Thu, Oct 24, 2013 at 11:09 AM, Chris Geer <ch...@cxtsoftware.com> wrote:
> On Sun, Oct 6, 2013 at 1:56 PM, Matt Franklin <m....@gmail.com>wrote:
>
>> On Tue, Sep 24, 2013 at 1:02 PM, Chris Geer <ch...@cxtsoftware.com>
>> wrote:
>>
>> > On Tue, Sep 24, 2013 at 9:54 AM, Michael Jett <mi...@gmail.com>
>> wrote:
>> >
>> > > On Mon, Sep 23, 2013 at 7:28 PM, Chris Geer <ch...@cxtsoftware.com>
>> > wrote:
>> > >
>> > > > Some additional comments below:
>> > > >
>> > > > On Sun, Sep 22, 2013 at 7:01 PM, Erin Noe-Payne <
>> > > erin.noe.payne@gmail.com
>> > > > >wrote:
>> > > >
>> > > > > Hey all, sorry I've been absent from this thread. I've caught up
>> now
>> > > > > and have a few thoughts:
>> > > > >
>> > > > > - As discussed before, the REST api should have CRUD endpoints
>> that
>> > > > > support all data retrieval and manipulation for any case that we
>> wish
>> > > > > for RAVE to allow. These endpoints should not be optimized for
>> > > > > specific client application / library needs.
>> > > > > - In addition we have some number of endpoints (currently one)
>> that
>> > > > > optimize for specific client needs. In this case we are talking
>> about
>> > > > > the pages for render endpoint, which serves 3 purposes:
>> > > > > 1) Filter to the set of pages by context and identifier
>> > > > > 2) Aggregate nested data for one read operation that gives all
>> data
>> > > > > necessary to render
>> > > > > 3) Attach special properties to regionWidgets needed to render
>> the
>> > > > > iframes (rpc tokens)
>> > > > > - The data structure we retrieve from the render endpoint should
>> not
>> > > > > be different from the models sent / consumed by the CRUD
>> endpoints.
>> > > > >
>> > > >
>> > > > So using your example above....the regionWidget CRUD model should
>> have
>> > > rpc
>> > > > token?
>> > > >
>> > > >
>> > > > > The render endpoint is read-only, so any data manipulation that
>> > > > > follows will be interacting with the CRUD endpoints. This is not
>> > > > > specific to angular - any client technology will be consuming the
>> > > > > render endpoint and feeding back that data in the case of updates.
>> > > > > Again, render endpoint is only filtering, aggregating nested data,
>> > > > > attaching render properties to regionWidget. Otherwise I don't
>> see a
>> > > > > reason the data models should differ.
>> > > > >
>> > > > > -A regionwidget should probably just have a mutable title
>> property.
>> > At
>> > > > > time of creation the title can be copied from the widget it is an
>> > > > > instance of, but afterwards it can be changed by the user. F.E if
>> I
>> > > > > have 3 instances of a map widget or a weather widget or something
>> on
>> > > > > my page, I may want to title them differently anyway.
>> > > > >
>> > > >
>> > > > I support this +1 as long as it's mutable.
>> > > >
>> > > > >
>> > > > > - I don't really know the best way to deal with relationships. My
>> > > > > instinct is that resources that have a one-to-X relationship
>> should
>> > > > > simply have the id of that relationship as a property. For
>> example,
>> > > > > each region has a pageId property. Each regionWidget has a pageId
>> and
>> > > > > regionId property. Then these relationships are atomically mutable
>> > via
>> > > > > a PUT request. Many-to-many relationships should be modeled via
>> their
>> > > > > own resource. So for example user-to-users friend should
>> correspond
>> > to
>> > > > > a friends resource that can be POSTed and DELETEd.
>> > > > >
>> > > >
>> > > > My opinion is our data model is bad on the server. A RegionWidget
>> > > shouldn't
>> > > > have a RegionID or PageID because they really shouldn't be mutable.
>> I
>> > > > realize this is a backend problem, not a REST problem exactly. I'm
>> just
>> > > > arguing we don't extend that to the REST interface so we can fix the
>> > > > backend problem later. I don't like a situation where you can have a
>> > url
>> > > > and data that can contradict.
>> > > >
>> > >
>> > > +1 - I think that this is a great point Chris. What planning or
>> > > coordination would it take to fix this backend problem?
>> > >
>> >
>> > I don't think fixing the backend problem is our primary concern as we
>> can
>> > work around it. My recommendation would be to design the REST API in
>> such a
>> > way as it makes the most sense, then we can fix the backend.
>> >
>> > >
>> > >
>> > > >
>> > > > > - Angular doesn't really care about this stuff. We have started
>> > > > > implementing according to what exists currently. If the api
>> changes,
>> > > > > then it will create some refactoring on the client. That's fine -
>> > > > > $resource is just json objects anyway, so it can handle it.
>> > However...
>> > > > > - We really just need consistency. My impression so far is that we
>> > > > > don't really have any practical expertise on building a robust
>> REST
>> > > > > api around a complicated data set. Short of that expertise
>> showing up
>> > > > > on the list, we really just need to rally around one strategy and
>> > > > > implement it consistently.
>> > > > >
>> > > >
>> > > > Well, that's not entirely true ;) At the end of the day, this data
>> > model
>> > > > really isn't all that complex (the rendering part). Page contains
>> 0-n
>> > > > Regions which contain 0-n Region Widgets. There is no reason why a
>> > > > RegionWidget needs a reference to the page it's on directly, that
>> > should
>> > > be
>> > > > derived by which resource it's retrieved from. It's that way on the
>> > > backend
>> > > > because our backed just mirrors the SQL table and it's foreign
>> keys. If
>> > > we
>> > > > had started this project with a NoSQL database, we never would have
>> had
>> > > > those attributes on those objects (probably wouldn't have used JPA
>> > > either).
>> > > >
>> > > > Using Mongo as an example, we never would have done this for a
>> page...
>> > > >
>> > > > {
>> > > > id: 123,
>> > > > regions: [
>> > > > {
>> > > > id: 332,
>> > > > page_id: 123,
>> > > > region_widgets: [
>> > > > {
>> > > > id: 454,
>> > > > region_id: 332
>> > > > }
>> > > > ]
>> > > > }
>> > > > ]
>> > > > }
>> > > >
>> > > > We would have stored page as a single document and not broken
>> Regions
>> > and
>> > > > RegionWidgets out into separate top level objects.
>> > > >
>> > > >
>> > > That makes sense. The REST interface would give us an opportunity to
>> > > consolidate and hide the relational keys, presenting objects in a
>> > "natural"
>> > > state. (i.e. pages, users) Rather than breaking it down into
>> proprietary
>> > > sub-object jargon. (i.e. RegionWidgets...)
>> > >
>> >
>> > Bingo!! Now we are on the same page.
>> >
>>
>> I think this all makes sense. The question now is if anyone has captured
>> it in Jira or started implementing it. I was looking at the Page rest
>> model object today in trunk and it is still the deep one we created to
>> support the full "rendered" object.
>>
>
> I don't believe anyone has done any work here yet. I will take a stab at
> something this week.
>
>>
>>
>> >
>> > >
>> > >
>> > > > Chris
>> > > >
>> > > >
>> > > > >
>> > > > > On Wed, Sep 18, 2013 at 1:29 PM, Chris Geer <
>> chris@cxtsoftware.com>
>> > > > wrote:
>> > > > > > Dan,
>> > > > > >
>> > > > > > My proposal is that on the server side, we do not add the extra
>> > > > > attributes
>> > > > > > to the REST model. Instead we have the render endpoint that will
>> > > return
>> > > > > the
>> > > > > > full object tree with the extra attributes. So on the server
>> side
>> > > there
>> > > > > > would be a different object like you said i.e.
>> > RenderedRegionWidget.
>> > > An
>> > > > > > alternative is to not create an objet and just build the JSON
>> > > directly
>> > > > > for
>> > > > > > the render endpoint.
>> > > > > >
>> > > > > > I'm also ok with the server ignoring unknown attributes on a
>> > > > save/update.
>> > > > > >
>> > > > > > Chris
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Sep 18, 2013 at 3:56 AM, Dan Gornstein <
>> dan@gornstein.com>
>> > > > > wrote:
>> > > > > >
>> > > > > >> Chris,
>> > > > > >>
>> > > > > >> I think I was a bit confused before and might have been
>> inaccurate
>> > > in
>> > > > > what
>> > > > > >> I was trying to say. I have been thinking about it a bit more
>> had
>> > a
>> > > > few
>> > > > > >> more questions for you.
>> > > > > >>
>> > > > > >> The endpoint which returns information for rendering a page is
>> a
>> > > > nested
>> > > > > >> object and I agree should be read only, meaning you cannot
>> post to
>> > > the
>> > > > > same
>> > > > > >> endpoint with the nested data model to save your object. In the
>> > > > angular
>> > > > > >> branch right now, when we hit this endpoint we break down the
>> > > response
>> > > > > into
>> > > > > >> their individual angular resource objects (page, regions,
>> > > > regionWidgets)
>> > > > > >> which point to the CRUD interfaces.
>> > > > > >>
>> > > > > >> The way the pages for render endpoint works right now is to use
>> > > > > pageService
>> > > > > >> to get the canonical model of a page and transform it into the
>> > REST
>> > > > > model
>> > > > > >> and send it through the DefaultRenderService to prepare the
>> > > > > regionWidgets.
>> > > > > >> This means it ends up returning the REST model of
>> RegionWidgets in
>> > > the
>> > > > > >> pages for render endpoint. So if you wanted to add the pageId
>> and
>> > > > title,
>> > > > > >> they would have to be properties on the RegionWidget REST
>> model.
>> > Are
>> > > > you
>> > > > > >> proposing that we make this pages for render endpoint be
>> compiled
>> > > with
>> > > > > >> different model objects other than the REST ones? Would there
>> be
>> > > > > something
>> > > > > >> along the lines of RegionWidgetRender, which has the extra
>> > > properties
>> > > > > but
>> > > > > >> does not have CRUD operations?
>> > > > > >>
>> > > > > >> If this was the case and we did not add pageId and title to the
>> > REST
>> > > > > models
>> > > > > >> of RegionWidgets, what will happen if we hit the pages for
>> render
>> > > > > endpoint
>> > > > > >> and in angular still broke them down into their individual
>> > resource
>> > > > > >> objects? Would you be allowed to save a RegionWidget to
>> > > > > >>
>> > > /pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id>
>> > > > > which
>> > > > > >> had title and pageId on it and the server would ignore it, or
>> > would
>> > > > you
>> > > > > >> expect the server to throw an error?
>> > > > > >>
>> > > > > >> Thanks,
>> > > > > >> Dan
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > > > >> On Mon, Sep 16, 2013 at 1:21 PM, Michael Jett <
>> > mikesjett@gmail.com>
>> > > > > wrote:
>> > > > > >>
>> > > > > >> > I'd like to pause this discussion for a brief moment, and
>> advise
>> > > > that
>> > > > > we
>> > > > > >> > watch this APIGee video.
>> > > > > >> >
>> > > http://www.youtube.com/watch?feature=player_embedded&v=QpAhXa12xvU
>> > > > > >> >
>> > > > > >> > This is something that Erin pointed us to early on, and
>> serves
>> > as
>> > > a
>> > > > > basis
>> > > > > >> > for a good RESTful architecture. There are too many good
>> points
>> > > made
>> > > > > that
>> > > > > >> > will allow our API to scale and be intuitive for other
>> > developers.
>> > > > > >> >
>> > > > > >> > It's very important that we are on the same page. I believe a
>> > lot
>> > > of
>> > > > > >> > "format" questions will be answered by following the APIGee
>> > > > > guidelines.
>> > > > > >> >
>> > > > > >> > - Mike
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > On Sun, Sep 15, 2013 at 6:30 PM, Chris Geer <
>> > > chris@cxtsoftware.com>
>> > > > > >> wrote:
>> > > > > >> >
>> > > > > >> > > Since we've been having a lot of discussions on data
>> > structures
>> > > > > lately
>> > > > > >> I
>> > > > > >> > > wanted to write down what my suggestions were. These aren't
>> > 100%
>> > > > > >> complete
>> > > > > >> > > examples but show the relationships
>> > > > > >> > >
>> > > > > >> > > CRUD Interfaces
>> > > > > >> > >
>> > > > > >> > > Page (/pages/<page-id>)
>> > > > > >> > > {
>> > > > > >> > > id: <id>,
>> > > > > >> > > owner: <owner>,
>> > > > > >> > > regionIds: [
>> > > > > >> > > <id>, <id>, <id> (region order is based on order in
>> > list,
>> > > > not
>> > > > > >> field
>> > > > > >> > > on region object)
>> > > > > >> > > ]
>> > > > > >> > > }
>> > > > > >> > >
>> > > > > >> > > Region (/pages/<page-id>/regions/<region-id>
>> > > > > >> > > {
>> > > > > >> > > id: <id>,
>> > > > > >> > > regionWidgetIds: [
>> > > > > >> > > <id>, <id>, <id> (widget order is based on order in
>> > list,
>> > > > not
>> > > > > >> field
>> > > > > >> > > on region widget object)
>> > > > > >> > > ]
>> > > > > >> > > }
>> > > > > >> > >
>> > > > > >> > > Region Widget
>> > > > > >> > >
>> > > > >
>> > (/pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id>
>> > > > > >> > > {
>> > > > > >> > > id: <id>,
>> > > > > >> > > widgetId: <widgetId>,
>> > > > > >> > > collapsed: <collapsed>,
>> > > > > >> > > ...
>> > > > > >> > > }
>> > > > > >> > >
>> > > > > >> > > Region Widget
>> > > > > >> > > Properties
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> (/pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id/properties/<propertyId>)
>> > > > > >> > > {
>> > > > > >> > > ....
>> > > > > >> > > }
>> > > > > >> > >
>> > > > > >> > > Widget (/widgets/<widget-id>
>> > > > > >> > > {
>> > > > > >> > > id: <id>
>> > > > > >> > > title: <title>
>> > > > > >> > > }
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > Render Interface (custom mime-type) (Read Only)
>> > > > > >> > >
>> > > > > >> > > /pages/<page-id>
>> > > > > >> > > {
>> > > > > >> > > id: <id>,
>> > > > > >> > > regions: [
>> > > > > >> > > {
>> > > > > >> > > id: <id>,
>> > > > > >> > > regionWidgets: [
>> > > > > >> > > {
>> > > > > >> > > id: <id>,
>> > > > > >> > > widgetId: <widgetId>,
>> > > > > >> > > title: <title>,
>> > > > > >> > > properties: [
>> > > > > >> > > {
>> > > > > >> > > key: <key>,
>> > > > > >> > > value: <value>
>> > > > > >> > > }
>> > > > > >> > > ]
>> > > > > >> > > }
>> > > > > >> > > ]
>> > > > > >> > > }
>> > > > > >> > > ]
>> > > > > >> > > }
>> > > > > >> > >
>> > > > > >> > > You should also be able to render sub elements below a
>> page so
>> > > for
>> > > > > >> > example,
>> > > > > >> > >
>> > > > > >> > > /pages/<page-id>/regions/<region-id> with the custom
>> mime-type
>> > > > would
>> > > > > >> > render
>> > > > > >> > > a single region.
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > Obviously there is still room for uncertainty in some
>> places.
>> > > For
>> > > > > >> > example,
>> > > > > >> > > what happens if your have a region with three region
>> widgets
>> > > then
>> > > > > you
>> > > > > >> > save
>> > > > > >> > > the region but only include two ids in the list?
>> Personally, I
>> > > > think
>> > > > > >> that
>> > > > > >> > > should delete the missing regionWidget because that list
>> > denotes
>> > > > > >> > ordering.
>> > > > > >> > > The reason I don't like an "order" attribute on the sub
>> > objects
>> > > is
>> > > > > that
>> > > > > >> > > what if you save two sub objects with the same order (which
>> > > would
>> > > > > >> happen
>> > > > > >> > if
>> > > > > >> > > you ever wanted to swap two objects in order because you
>> have
>> > to
>> > > > > update
>> > > > > >> > > them then save each one so they would have the same order
>> at
>> > > least
>> > > > > >> > > momentarily on the server)?
>> > > > > >> > >
>> > > > > >> > > Anyway, my 2-cents.
>> > > > > >> > >
>> > > > > >> > > Chris
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Rave Data Structures
Posted by Chris Geer <ch...@cxtsoftware.com>.
On Sun, Oct 6, 2013 at 1:56 PM, Matt Franklin <m....@gmail.com>wrote:
> On Tue, Sep 24, 2013 at 1:02 PM, Chris Geer <ch...@cxtsoftware.com> wrote:
>
> > On Tue, Sep 24, 2013 at 9:54 AM, Michael Jett <mi...@gmail.com>
> wrote:
> >
> > > On Mon, Sep 23, 2013 at 7:28 PM, Chris Geer <ch...@cxtsoftware.com>
> > wrote:
> > >
> > > > Some additional comments below:
> > > >
> > > > On Sun, Sep 22, 2013 at 7:01 PM, Erin Noe-Payne <
> > > erin.noe.payne@gmail.com
> > > > >wrote:
> > > >
> > > > > Hey all, sorry I've been absent from this thread. I've caught up
> now
> > > > > and have a few thoughts:
> > > > >
> > > > > - As discussed before, the REST api should have CRUD endpoints that
> > > > > support all data retrieval and manipulation for any case that we
> wish
> > > > > for RAVE to allow. These endpoints should not be optimized for
> > > > > specific client application / library needs.
> > > > > - In addition we have some number of endpoints (currently one) that
> > > > > optimize for specific client needs. In this case we are talking
> about
> > > > > the pages for render endpoint, which serves 3 purposes:
> > > > > 1) Filter to the set of pages by context and identifier
> > > > > 2) Aggregate nested data for one read operation that gives all
> data
> > > > > necessary to render
> > > > > 3) Attach special properties to regionWidgets needed to render
> the
> > > > > iframes (rpc tokens)
> > > > > - The data structure we retrieve from the render endpoint should
> not
> > > > > be different from the models sent / consumed by the CRUD endpoints.
> > > > >
> > > >
> > > > So using your example above....the regionWidget CRUD model should
> have
> > > rpc
> > > > token?
> > > >
> > > >
> > > > > The render endpoint is read-only, so any data manipulation that
> > > > > follows will be interacting with the CRUD endpoints. This is not
> > > > > specific to angular - any client technology will be consuming the
> > > > > render endpoint and feeding back that data in the case of updates.
> > > > > Again, render endpoint is only filtering, aggregating nested data,
> > > > > attaching render properties to regionWidget. Otherwise I don't see
> a
> > > > > reason the data models should differ.
> > > > >
> > > > > -A regionwidget should probably just have a mutable title property.
> > At
> > > > > time of creation the title can be copied from the widget it is an
> > > > > instance of, but afterwards it can be changed by the user. F.E if I
> > > > > have 3 instances of a map widget or a weather widget or something
> on
> > > > > my page, I may want to title them differently anyway.
> > > > >
> > > >
> > > > I support this +1 as long as it's mutable.
> > > >
> > > > >
> > > > > - I don't really know the best way to deal with relationships. My
> > > > > instinct is that resources that have a one-to-X relationship should
> > > > > simply have the id of that relationship as a property. For example,
> > > > > each region has a pageId property. Each regionWidget has a pageId
> and
> > > > > regionId property. Then these relationships are atomically mutable
> > via
> > > > > a PUT request. Many-to-many relationships should be modeled via
> their
> > > > > own resource. So for example user-to-users friend should correspond
> > to
> > > > > a friends resource that can be POSTed and DELETEd.
> > > > >
> > > >
> > > > My opinion is our data model is bad on the server. A RegionWidget
> > > shouldn't
> > > > have a RegionID or PageID because they really shouldn't be mutable. I
> > > > realize this is a backend problem, not a REST problem exactly. I'm
> just
> > > > arguing we don't extend that to the REST interface so we can fix the
> > > > backend problem later. I don't like a situation where you can have a
> > url
> > > > and data that can contradict.
> > > >
> > >
> > > +1 - I think that this is a great point Chris. What planning or
> > > coordination would it take to fix this backend problem?
> > >
> >
> > I don't think fixing the backend problem is our primary concern as we can
> > work around it. My recommendation would be to design the REST API in
> such a
> > way as it makes the most sense, then we can fix the backend.
> >
> > >
> > >
> > > >
> > > > > - Angular doesn't really care about this stuff. We have started
> > > > > implementing according to what exists currently. If the api
> changes,
> > > > > then it will create some refactoring on the client. That's fine -
> > > > > $resource is just json objects anyway, so it can handle it.
> > However...
> > > > > - We really just need consistency. My impression so far is that we
> > > > > don't really have any practical expertise on building a robust REST
> > > > > api around a complicated data set. Short of that expertise showing
> up
> > > > > on the list, we really just need to rally around one strategy and
> > > > > implement it consistently.
> > > > >
> > > >
> > > > Well, that's not entirely true ;) At the end of the day, this data
> > model
> > > > really isn't all that complex (the rendering part). Page contains 0-n
> > > > Regions which contain 0-n Region Widgets. There is no reason why a
> > > > RegionWidget needs a reference to the page it's on directly, that
> > should
> > > be
> > > > derived by which resource it's retrieved from. It's that way on the
> > > backend
> > > > because our backed just mirrors the SQL table and it's foreign keys.
> If
> > > we
> > > > had started this project with a NoSQL database, we never would have
> had
> > > > those attributes on those objects (probably wouldn't have used JPA
> > > either).
> > > >
> > > > Using Mongo as an example, we never would have done this for a
> page...
> > > >
> > > > {
> > > > id: 123,
> > > > regions: [
> > > > {
> > > > id: 332,
> > > > page_id: 123,
> > > > region_widgets: [
> > > > {
> > > > id: 454,
> > > > region_id: 332
> > > > }
> > > > ]
> > > > }
> > > > ]
> > > > }
> > > >
> > > > We would have stored page as a single document and not broken Regions
> > and
> > > > RegionWidgets out into separate top level objects.
> > > >
> > > >
> > > That makes sense. The REST interface would give us an opportunity to
> > > consolidate and hide the relational keys, presenting objects in a
> > "natural"
> > > state. (i.e. pages, users) Rather than breaking it down into
> proprietary
> > > sub-object jargon. (i.e. RegionWidgets...)
> > >
> >
> > Bingo!! Now we are on the same page.
> >
>
> I think this all makes sense. The question now is if anyone has captured
> it in Jira or started implementing it. I was looking at the Page rest
> model object today in trunk and it is still the deep one we created to
> support the full "rendered" object.
>
I don't believe anyone has done any work here yet. I will take a stab at
something this week.
>
>
> >
> > >
> > >
> > > > Chris
> > > >
> > > >
> > > > >
> > > > > On Wed, Sep 18, 2013 at 1:29 PM, Chris Geer <chris@cxtsoftware.com
> >
> > > > wrote:
> > > > > > Dan,
> > > > > >
> > > > > > My proposal is that on the server side, we do not add the extra
> > > > > attributes
> > > > > > to the REST model. Instead we have the render endpoint that will
> > > return
> > > > > the
> > > > > > full object tree with the extra attributes. So on the server side
> > > there
> > > > > > would be a different object like you said i.e.
> > RenderedRegionWidget.
> > > An
> > > > > > alternative is to not create an objet and just build the JSON
> > > directly
> > > > > for
> > > > > > the render endpoint.
> > > > > >
> > > > > > I'm also ok with the server ignoring unknown attributes on a
> > > > save/update.
> > > > > >
> > > > > > Chris
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 18, 2013 at 3:56 AM, Dan Gornstein <
> dan@gornstein.com>
> > > > > wrote:
> > > > > >
> > > > > >> Chris,
> > > > > >>
> > > > > >> I think I was a bit confused before and might have been
> inaccurate
> > > in
> > > > > what
> > > > > >> I was trying to say. I have been thinking about it a bit more
> had
> > a
> > > > few
> > > > > >> more questions for you.
> > > > > >>
> > > > > >> The endpoint which returns information for rendering a page is a
> > > > nested
> > > > > >> object and I agree should be read only, meaning you cannot post
> to
> > > the
> > > > > same
> > > > > >> endpoint with the nested data model to save your object. In the
> > > > angular
> > > > > >> branch right now, when we hit this endpoint we break down the
> > > response
> > > > > into
> > > > > >> their individual angular resource objects (page, regions,
> > > > regionWidgets)
> > > > > >> which point to the CRUD interfaces.
> > > > > >>
> > > > > >> The way the pages for render endpoint works right now is to use
> > > > > pageService
> > > > > >> to get the canonical model of a page and transform it into the
> > REST
> > > > > model
> > > > > >> and send it through the DefaultRenderService to prepare the
> > > > > regionWidgets.
> > > > > >> This means it ends up returning the REST model of RegionWidgets
> in
> > > the
> > > > > >> pages for render endpoint. So if you wanted to add the pageId
> and
> > > > title,
> > > > > >> they would have to be properties on the RegionWidget REST model.
> > Are
> > > > you
> > > > > >> proposing that we make this pages for render endpoint be
> compiled
> > > with
> > > > > >> different model objects other than the REST ones? Would there be
> > > > > something
> > > > > >> along the lines of RegionWidgetRender, which has the extra
> > > properties
> > > > > but
> > > > > >> does not have CRUD operations?
> > > > > >>
> > > > > >> If this was the case and we did not add pageId and title to the
> > REST
> > > > > models
> > > > > >> of RegionWidgets, what will happen if we hit the pages for
> render
> > > > > endpoint
> > > > > >> and in angular still broke them down into their individual
> > resource
> > > > > >> objects? Would you be allowed to save a RegionWidget to
> > > > > >>
> > > /pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id>
> > > > > which
> > > > > >> had title and pageId on it and the server would ignore it, or
> > would
> > > > you
> > > > > >> expect the server to throw an error?
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Dan
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Mon, Sep 16, 2013 at 1:21 PM, Michael Jett <
> > mikesjett@gmail.com>
> > > > > wrote:
> > > > > >>
> > > > > >> > I'd like to pause this discussion for a brief moment, and
> advise
> > > > that
> > > > > we
> > > > > >> > watch this APIGee video.
> > > > > >> >
> > > http://www.youtube.com/watch?feature=player_embedded&v=QpAhXa12xvU
> > > > > >> >
> > > > > >> > This is something that Erin pointed us to early on, and serves
> > as
> > > a
> > > > > basis
> > > > > >> > for a good RESTful architecture. There are too many good
> points
> > > made
> > > > > that
> > > > > >> > will allow our API to scale and be intuitive for other
> > developers.
> > > > > >> >
> > > > > >> > It's very important that we are on the same page. I believe a
> > lot
> > > of
> > > > > >> > "format" questions will be answered by following the APIGee
> > > > > guidelines.
> > > > > >> >
> > > > > >> > - Mike
> > > > > >> >
> > > > > >> >
> > > > > >> > On Sun, Sep 15, 2013 at 6:30 PM, Chris Geer <
> > > chris@cxtsoftware.com>
> > > > > >> wrote:
> > > > > >> >
> > > > > >> > > Since we've been having a lot of discussions on data
> > structures
> > > > > lately
> > > > > >> I
> > > > > >> > > wanted to write down what my suggestions were. These aren't
> > 100%
> > > > > >> complete
> > > > > >> > > examples but show the relationships
> > > > > >> > >
> > > > > >> > > CRUD Interfaces
> > > > > >> > >
> > > > > >> > > Page (/pages/<page-id>)
> > > > > >> > > {
> > > > > >> > > id: <id>,
> > > > > >> > > owner: <owner>,
> > > > > >> > > regionIds: [
> > > > > >> > > <id>, <id>, <id> (region order is based on order in
> > list,
> > > > not
> > > > > >> field
> > > > > >> > > on region object)
> > > > > >> > > ]
> > > > > >> > > }
> > > > > >> > >
> > > > > >> > > Region (/pages/<page-id>/regions/<region-id>
> > > > > >> > > {
> > > > > >> > > id: <id>,
> > > > > >> > > regionWidgetIds: [
> > > > > >> > > <id>, <id>, <id> (widget order is based on order in
> > list,
> > > > not
> > > > > >> field
> > > > > >> > > on region widget object)
> > > > > >> > > ]
> > > > > >> > > }
> > > > > >> > >
> > > > > >> > > Region Widget
> > > > > >> > >
> > > > >
> > (/pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id>
> > > > > >> > > {
> > > > > >> > > id: <id>,
> > > > > >> > > widgetId: <widgetId>,
> > > > > >> > > collapsed: <collapsed>,
> > > > > >> > > ...
> > > > > >> > > }
> > > > > >> > >
> > > > > >> > > Region Widget
> > > > > >> > > Properties
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> (/pages/<page-id>/regions/<region-id>/regionwidgets/<region-widget-id/properties/<propertyId>)
> > > > > >> > > {
> > > > > >> > > ....
> > > > > >> > > }
> > > > > >> > >
> > > > > >> > > Widget (/widgets/<widget-id>
> > > > > >> > > {
> > > > > >> > > id: <id>
> > > > > >> > > title: <title>
> > > > > >> > > }
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > Render Interface (custom mime-type) (Read Only)
> > > > > >> > >
> > > > > >> > > /pages/<page-id>
> > > > > >> > > {
> > > > > >> > > id: <id>,
> > > > > >> > > regions: [
> > > > > >> > > {
> > > > > >> > > id: <id>,
> > > > > >> > > regionWidgets: [
> > > > > >> > > {
> > > > > >> > > id: <id>,
> > > > > >> > > widgetId: <widgetId>,
> > > > > >> > > title: <title>,
> > > > > >> > > properties: [
> > > > > >> > > {
> > > > > >> > > key: <key>,
> > > > > >> > > value: <value>
> > > > > >> > > }
> > > > > >> > > ]
> > > > > >> > > }
> > > > > >> > > ]
> > > > > >> > > }
> > > > > >> > > ]
> > > > > >> > > }
> > > > > >> > >
> > > > > >> > > You should also be able to render sub elements below a page
> so
> > > for
> > > > > >> > example,
> > > > > >> > >
> > > > > >> > > /pages/<page-id>/regions/<region-id> with the custom
> mime-type
> > > > would
> > > > > >> > render
> > > > > >> > > a single region.
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > Obviously there is still room for uncertainty in some
> places.
> > > For
> > > > > >> > example,
> > > > > >> > > what happens if your have a region with three region widgets
> > > then
> > > > > you
> > > > > >> > save
> > > > > >> > > the region but only include two ids in the list?
> Personally, I
> > > > think
> > > > > >> that
> > > > > >> > > should delete the missing regionWidget because that list
> > denotes
> > > > > >> > ordering.
> > > > > >> > > The reason I don't like an "order" attribute on the sub
> > objects
> > > is
> > > > > that
> > > > > >> > > what if you save two sub objects with the same order (which
> > > would
> > > > > >> happen
> > > > > >> > if
> > > > > >> > > you ever wanted to swap two objects in order because you
> have
> > to
> > > > > update
> > > > > >> > > them then save each one so they would have the same order at
> > > least
> > > > > >> > > momentarily on the server)?
> > > > > >> > >
> > > > > >> > > Anyway, my 2-cents.
> > > > > >> > >
> > > > > >> > > Chris
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>