You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@olingo.apache.org by Tom van Wietmarschen <aa...@gmail.com> on 2016/03/02 12:54:27 UTC

Stream properties

I'm trying to add a property of type EdmPrimitiveTypeKind.Stream to my entity, I can set it to this type in the metadata, but how do I provide the actual content ? There seems to be no documentation whatsoever on this. My Entity contains several images that I want to be able to access, preferrably I'd like to be able to provide different image formats too depending on accept headers or maybe an URL parameter.

Is there a place where I can find this kind of information ? The javadocs are pretty much content-free, and the tutorial does not cover this and many other cases.

/Tom

Re: Stream properties

Posted by Ramesh Reddy <ra...@redhat.com>.

You are correct that there is no indexed access to the collection (yet, being considered 4.01), however per spec the access to the document entity should reply document's properties, including the Edm.Stream collection. However note that in that case, collection is returning the your image metadata along with the URL for the image(s). Based on that payload details, you need to access the images. The idea behind is, streams may hosted in a separate container where URL to access them needs to be user defined, so it gives you that flexibility. If you want that can be 

/document(0)/image1 

you can design that way in your service implementation. 

Coming to your criticism about Olingo, it is a library that gives you tools like URL parser, Serializers, metadata generation, client access so on. There are few examples on Olingo site https://olingo.apache.org/doc/odata4/index.html , there are few more in code. I remember seeing some code example about streams couple days ago. As per it working as a framework, the example link I gave you is the going in that direction. I have written plenty of OData code using Olingo for http;//teiid.org , you can check our git repo for it. This is community project your feedback and contributions are most welcome. 

Thanks. 

Ramesh.. 

----- Original Message -----

> > On 08 Mar 2016, at 16:21, Ramesh Reddy < rareddy@redhat.com > wrote:
> 

> > EntityType {Document} {
> 
> > id Edm.String,
> 
> > description Edm.String,
> 
> > Collection of ComplexType {Image} {
> 
> > name Edm.String,
> 
> > descripiton Edm.String,
> 
> > contents Edm.Stream
> 
> > }
> 
> > }
> 

> So this is not possible after all, because after going through the OData spec
> there doesn't seem to be any way to address a member of a collection. That
> is: it is simply not possible to construct an URL that points to 'contents'
> (or any other member) of the Nth element in the collection in the above
> structure. So basically if you put *anything* into a collection, you can no
> longer directly address it. This also means I cannot construct a read-URL
> for the stream.

> I tried solving this by making a bound function that takes an element number
> and returns that element, which while ugly *should* work. I've declared it
> in the EDM, but according to the spec (Odata v4 JSON format chapter 15) a
> bound function should also be advertised in the Entity, except it's not. I
> expected this to be automatic (considering at that point I've given Olingo
> all the information it needs to do so) but it doesn't, and I can't find
> *any* information on how to do it manually. I assume it has something to do
> with the list of Operations in Entity, but how is not clear. None of this is
> documented anywhere. I've googled, searched the mailing lists, looked for
> testcases in git that happen to test this functionality (there aren't any),
> looked through the source for any hints on how this is supposed to be done,
> but couldn't find a thing.

> > Yes, there is bit of learning curve.
> 

> I'm sorry to say that at this point I'm completely fed up with Olingo. This
> has nothing to do with a learning curve, a learning curve requires there to
> be material to learn from, and there simply is none. The Javadocs may as
> well not exist, they are obviously auto-generated and zero effort was put
> into them.

> By now, I've wasted weeks on what should have been a 2 day task. I could
> probably have implemented an OData interface for my service from scratch
> using just plain-old servlets in less time than it took me to get to this
> point using Olingo. Right now, I'm not sure if I should continue pouring
> more effort into this. I have to conclude that Olingo is impossible to use
> because it needs documentation for anything but very trivial cases that
> happen to be similar to the tutorial and that documentation does not exist
> anywhere.

> /Tom

Re: Stream properties

Posted by Tom van Wietmarschen <aa...@gmail.com>.

> On 08 Mar 2016, at 16:21, Ramesh Reddy <ra...@redhat.com> wrote:
> 
> EntityType {Document} {
>    id Edm.String,
>    description Edm.String,
>    Collection of ComplexType {Image} {
>         name Edm.String,
>         descripiton Edm.String,
>         contents Edm.Stream
>    }
> }

So this is not possible after all, because after going through the OData spec there doesn't seem to be any way to address a member of a collection. That is: it is simply not possible to construct an URL that points to 'contents' (or any other member) of the Nth element in the collection in the above structure.  So basically if you put *anything* into a collection, you can no longer directly address it. This also means I cannot construct a read-URL for the stream. 

I tried solving this by making a bound function that takes an element number and returns that element, which while ugly *should* work. I've declared it in the EDM, but according to the spec (Odata v4 JSON format chapter 15) a bound function should also be advertised in the Entity, except it's not. I expected this to be automatic (considering at that point I've given Olingo all the information it needs to do so) but it doesn't, and I can't find *any* information on how to do it manually. I assume it has something to do with the list of Operations in Entity, but how is not clear. None of this is documented anywhere. I've googled, searched the mailing lists, looked for testcases in git that happen to test this functionality (there aren't any), looked through the source for any hints on how this is supposed to be done, but couldn't find a thing. 

> Yes, there is bit of learning curve.

I'm sorry to say that at this point I'm completely fed up with Olingo. This has nothing to do with a learning curve, a learning curve requires there to be material to learn from, and there simply is none. The Javadocs may as well not exist, they are obviously auto-generated and zero effort was put into them. 

By now, I've wasted weeks on what should have been a 2 day task. I could probably have implemented an OData interface for my service from scratch using just plain-old servlets in less time than it took me to get to this point using Olingo. Right now, I'm not sure if I should continue pouring more effort into this. I have to conclude that Olingo is impossible to use because it needs documentation for anything but very trivial cases that happen to be similar to the tutorial and that documentation does not exist anywhere. 

/Tom

Re: Stream properties

Posted by Tom van Wietmarschen <aa...@gmail.com>.

> On 08 Mar 2016, at 16:21, Ramesh Reddy <ra...@redhat.com> wrote:
> 
> Tom,
> 
> First of all  OData Spec != Olingo, OData specification outlines all the behavior, Olingo is a Java based library which tried to implement a framework that can be used by a Java developer.

I understand the distinction, which is why I was originally looking for Olingo documentation instead of OData documentation. There seem to be a lot of implementation-details that are not documented at all. I would really prefer to be able to RTFM instead of asking dumb questions here but unfortunately it mostly turned into UTSL instead. Especially since it doesn't seem to handle errors graciously yet. Apart from it swallowing exceptions without any log statement whatsoever, the data I get from the debug interface is less than helpful. E.g. if you forget to set the name on an Entity, instead of an error telling me I have forgotten to set a required property, I get a NPE from somewhere deep inside the serializer that I have to track down to the root cause. Very time consuming.  

> Olingo implementation is not complete, that is why you will see "not implemented" errors. But we are working towards providing all the features. But OData V4 specification is quite complete and extensive.

I was misled a bit by the version number, Olingo v4.1.0 would imply it's feature-complete and mature. I would expect anything above v1.0.0 to be feature-complete at least. Yes I know now it refers to OData 4 , but for someone just getting started on this it can be confusing. 

> EntityTypes are top level constructs, EntityType can have other EntityTypes but not as properties, but as Navigation property, think like they define relationships. ComplexTypes are just as name specifies a complex property. May be thinking OData as JPA model helps.

I was discussing this with a colleague who has a .NET background, it seems like a lot of the .NET persistence layer is leaking through the spec. Initially, I was thinking of Entities more like Objects than something like a JPA entity. In the same way, a ComplexValue would just be an Entity that can't be addressed directly. (I expected Entity to be like a subclass of ComplexValue). 

> I agree that MediaEntity and Stream properties are confusing. The hard rule is if Entity has implicit stream then that is MediaEntity, which can have one single stream and have simple access mechanisms. Where as you want to define as properties of Entity then use stream property, and access is similar to accessing a property. I suggest taking looking at this another example we have [1], which is little bit more straight forward and does lot more lifting of most of the boiler plate code. 

I had a look, took me a while to figure it out because it is a completely different way of doing things compared to the tutorials and documentation on the Olingo site. 

It makes a bit more sense now. In you example I see a 'Picture' property that has some annotations attached to it and if I interpret it correctly reading the raw value of the property as a single value property (through PrimitiveValueProcessor) will output the stream. 

> Now coming to your example, can you tell me where you saw ComplexTypes do not support stream properties?

It does support stream properties, I was referring to the ComplexType itself having hasStream = true like an Entity. I would like to have something like this:

EntityType (Document) {
  id Edm.String
  Collection of ComplexType {Image, hasStream=true} {
    width Edm.Int32,
    height Edm.Int32,
    bitDepth Edm.Int32
  }
}

And then be able to do something like:  GET /Document('id')/image[0]/$value

> I would have thought a entity type like below should work

Your example will work for me, but I would have preferred the variant above. 
.
> 
> [1] https://github.com/apache/olingo-odata4/tree/master/lib/server-core-ext/src/test/java/org/apache/olingo/server/example <https://github.com/apache/olingo-odata4/tree/master/lib/server-core-ext/src/test/java/org/apache/olingo/server/example>
Interesting example, that clears a lot up for me. 

Do I have to set the annotation for the read link or will it generate it itself ? Or, I guess what I'm actually trying to find out: why is there a read/write link annotation at all, considering there isn't one for any of the other types. 

/Tom

Re: Stream properties

Posted by Ramesh Reddy <ra...@redhat.com>.

Tom,

First of all OData Spec != Olingo, OData specification outlines all the behavior, Olingo is a Java based library which tried to implement a framework that can be used by a Java developer. Olingo implementation is not complete, that is why you will see "not implemented" errors. But we are working towards providing all the features. But OData V4 specification is quite complete and extensive.

EntityTypes are top level constructs, EntityType can have other EntityTypes but not as properties, but as Navigation property, think like they define relationships. ComplexTypes are just as name specifies a complex property. May be thinking OData as JPA model helps.

I agree that MediaEntity and Stream properties are confusing. The hard rule is if Entity has implicit stream then that is MediaEntity, which can have one single stream and have simple access mechanisms. Where as you want to define as properties of Entity then use stream property, and access is similar to accessing a property. I suggest taking looking at this another example we have [1], which is little bit more straight forward and does lot more lifting of most of the boiler plate code.

Now coming to your example, can you tell me where you saw ComplexTypes do not support stream properties? I would have thought a entity type like below should work

EntityType {Document} {
id Edm.String,
description Edm.String,
Collection of ComplexType {Image} {
name Edm.String,
descripiton Edm.String,
contents Edm.Stream
}
}

Yes, there is bit of learning curve.

Ramesh..

[1] https://github.com/apache/olingo-odata4/tree/master/lib/server-core-ext/src/test/java/org/apache/olingo/server/example
----- Original Message -----

Ramesh,

Thanks for your reply.

You should refer to the OData specification for finer details, see http://odata.org

I did have a look at the specs, which was the source of my confusion. There are two kinds of Stream, one associated with an entity (hasStream = true) and one associated with a property (Type = PrimitiveTypeKind.Stream). Entities with streams are handled through Olingo (through MediaEntityProcessor) so I was looking for an equivalent for stream properties (something like MediaPropertyProcessor), which doesn't seem to exist. Instead it requires you to provide a URI to ... where exactly ? Another servlet ? So basically there are two ways to deal with streams in Olingo: one is integrated with Olingo itself and the other one requires a completely separate servlet to be built to deal with it. Am I correctly interpreting the spec and if so am I wrong in thinking this is kind of weird and inconsistent ? Same goes for ComplexValue v.s Entity which (correct me if I'm wrong) is conceptually just a ComplexValue with a primary key. They have very different APIs (e.g. addProperty() vs. getProperties().add()) despite being conceptually very similar. It feels weird.

To avoid a case of "x-y problem", here is what I'm trying to achieve; I'm starting to wonder if this (fairly trivial case) is impossible to model in OData:

I have a collection of documents, each document contains zero or more Images. These images are embedded in the documents, they are identified by documentId + imageNr. For each image I want to provide metadata and I want to be able to retrieve each image individually. Ideally, I'd access the document at <root>/Document('id'), just the image-metadata at something equivalent to <root>/Document('id')/image[0], and the actual image at something like <root>/Document('id')/image[0]/$value.

Since entities support streams, adding a property of type 'Entity' for the metadata seems to be the obvious solution, but that gives a 'not yet supported' error. Even weirder, the OData specs imply that a property cannot be of type Entity (so why is it 'not yet' supported, will it be in OData v5?). Although a navigation property would probably allow me to do this, that seems incorrect as I'm not navigating to a different entity but to a member of the current entity. ComplexValue makes more sense for what I'm trying to do, but that doesn't support Streams (why the hell not? Is it not just an Entity without a key ?). Stream properties could be a solution, but those live outside the OData servlet and it would be kinda ugly in that there is no consistent way to address a document if some members of the document use a completely different path.

There are some other ways I can think of of exposing the images, but they are all just as ugly and feel like they are not how it's supposed to be done. Can you or anyone else tell me what the 'odata-way' of doing this is ? The OData spec does not go into this at all.

I'm getting the feeling I'm missing the big picture here. Basically all the documentation I've found on OData and Olingo focusses on the 'how', not on the 'why'. Is there some kind of resource that explains the design decisions taken, which trade-offs where chosen and the basic high-level architecture and concepts, like why is Entity so different from ComplexValue ? RIght now, it feels to me the OData spec has no conceptual-integrity and considering how much effort appears to have gone into the odata spec it seems more likely to be a lack of information on my side than a problem with the spec.

Sincerely,
Tom

</blockquote>

Re: Stream properties

Posted by Tom van Wietmarschen <aa...@gmail.com>.

Ramesh,

Thanks for your reply.

> You should refer to the OData specification for finer details, see http://odata.org <http://odata.org/>

To avoid a case of "x-y problem", here is what I'm trying to achieve; I'm starting to wonder if this (fairly trivial case) is impossible to model in OData:

Sincerely,
Tom

Re: Stream properties

Posted by Ramesh Reddy <ra...@redhat.com>.

Tom,

You should refer to the OData specification for finer details, see http://odata.org

Ramesh..
http://teiid.org

----- Original Message -----
> I'm trying to add a property of type EdmPrimitiveTypeKind.Stream to my
> entity, I can set it to this type in the metadata, but how do I provide the
> actual content ? There seems to be no documentation whatsoever on this. My
> Entity contains several images that I want to be able to access, preferrably
> I'd like to be able to provide different image formats too depending on
> accept headers or maybe an URL parameter.
> 
> Is there a place where I can find this kind of information ? The javadocs are
> pretty much content-free, and the tutorial does not cover this and many
> other cases.
> 
> /Tom