You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Pradeep Fernando <pr...@gmail.com> on 2014/04/18 08:53:38 UTC

Validating Inputs to REST API

Hi Devs,

I started on the $subject, since there was a issue marked as critical.

https://issues.apache.org/jira/browse/STRATOS-466 (patch attached)

I was able to provide a fix. Now the incoming messages (JSON) are validated
against xmlSchema file that we provide and if there is an exception while
data binding due to wrong message format, we spit out a json error message.

This fix however affects all our existing clients. Hence there is a strong
possibility of regression issues. Hence I would suggest to incoperate this
fix in to next release, and lower the priority of the fix.

Thanks,
--Pradeep

Re: Validating Inputs to REST API

Posted by Nirmal Fernando <ni...@gmail.com>.
On Mon, Sep 1, 2014 at 2:25 PM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:

> Hi,
>
> Bringing up this thread back since validation is broken in master
> branch. I've fixed validations schema to reflect the recent changes
> done to auto-scaling policy (in STRATOS-645).
>
> @Nirmal, can you remember what policies were not covered in the
> validation schema? I did some cross checking but I guess I'm missing
> something here.
>

Unfortunately no Akila.. I remember pointing out few, but have lost track
of it. I guess someone just need to put bit care onto validation schemas
and fix it :-)

>
> Thanks.
>
> On Wed, May 7, 2014 at 4:11 AM, Nirmal Fernando <ni...@gmail.com>
> wrote:
> > Hi Pradeep,
> >
> > XSD schema seems validating only few levels of the policy files. Is there
> > any reason for not validating full config?
> >
> >
> > On Tue, Apr 29, 2014 at 10:47 AM, Pradeep Fernando <pr...@gmail.com>
> > wrote:
> >>
> >> Hi Akila,
> >>
> >>
> >> On Mon, Apr 28, 2014 at 1:24 PM, Akila Ravihansa Perera
> >> <ra...@wso2.com> wrote:
> >>>
> >>> Hi Pradeep,
> >>>
> >>> I have some concerns regarding the schema.
> >>>
> >>> 1. In partition deployment, do we really need to make it mandatory to
> >>> include partitionMin and partitionMax?
> >>> IMO, minOccurs should be 0 for these two properties since this can be
> >>> overridden in the deployment definition
> >>
> >>
> >> +1. Please go ahead and change
> >>>
> >>>
> >>> 2. What is the need for having a sequence in properties definition?
> >>> For eg - In cartridge definition if the user don't enter the values in
> >>> the order that you have defined in the schema it won't validate. IMO,
> >>> we should use <xs:choice> instead of  <xs:sequence> when describing
> >>> properties.
> >>
> >>
> >> Sorry i did not get you. I used xs:all when definining property arrays.
> >>
> >>>
> >>>
> >>> Thanks.
> >>>
> >>> On Wed, Apr 23, 2014 at 4:22 PM, Pradeep Fernando <pradeepfn@gmail.com
> >
> >>> wrote:
> >>> > Hi devs,
> >>> >
> >>> > Now the API spits error messages irrespective of the backend
> exception.
> >>> > It
> >>> > sends the JSON error message even if there is an Nullpointer in our
> >>> > backend.
> >>> > Writing a exception mapper for throwable did the trick.
> >>> >
> >>> > Here is the schema I came up with for validating input. I derived it
> >>> > from
> >>> > the sample inputs i have. Please review it. Pretty sure there is
> space
> >>> > for
> >>> > improvements particularly min occurs.
> >>> >
> >>> >
> >>> > <?xml version="1.0" encoding="UTF-8" ?>
> >>> > <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
> >>> >     <xs:element name="partition">
> >>> >         <xs:complexType>
> >>> >             <xs:sequence>
> >>> >                 <xs:element name="id" type="xs:string" minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="provider" type="xs:string"
> >>> > minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="property" minOccurs="1"
> >>> > maxOccurs="unbounded">
> >>> >                     <xs:complexType>
> >>> >                         <xs:all>
> >>> >                             <xs:element name="name" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="value" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                         </xs:all>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >                 <xs:element name="partitionMin" type="xs:int"
> >>> > minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="partitionMax" type="xs:int"
> >>> > minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >             </xs:sequence>
> >>> >         </xs:complexType>
> >>> >     </xs:element>
> >>> >
> >>> >     <xs:element name="autoscalePolicy">
> >>> >         <xs:complexType>
> >>> >             <xs:sequence>
> >>> >                 <xs:element name="id" type="xs:string" minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="loadThresholds" minOccurs="1"
> >>> > maxOccurs="1" nillable="false">
> >>> >                     <xs:complexType>
> >>> >                         <xs:all>
> >>> >                             <xs:element name="requestsInFlight"
> >>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="memoryConsumption"
> >>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="loadAverage"
> >>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                         </xs:all>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >             </xs:sequence>
> >>> >         </xs:complexType>
> >>> >     </xs:element>
> >>> >
> >>> >     <xs:element name="deploymentPolicy">
> >>> >         <xs:complexType>
> >>> >             <xs:sequence>
> >>> >                 <xs:element name="id" type="xs:string" maxOccurs="1"
> >>> > minOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="partitionGroup">
> >>> >                     <xs:complexType>
> >>> >                         <xs:sequence>
> >>> >                             <xs:element name="id" type="xs:string"
> >>> > maxOccurs="1" minOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="partitionAlgo"
> >>> > minOccurs="1"
> >>> > maxOccurs="1" nillable="false">
> >>> >                                 <xs:simpleType>
> >>> >                                     <xs:restriction base="xs:string">
> >>> >                                         <xs:enumeration
> >>> > value="round-robin"/>
> >>> >                                         <xs:enumeration
> >>> > value="one-after-another"/>
> >>> >                                     </xs:restriction>
> >>> >                                 </xs:simpleType>
> >>> >                             </xs:element>
> >>> >                             <xs:element name="partition"
> minOccurs="1"
> >>> > maxOccurs="unbounded">
> >>> >                                 <xs:complexType>
> >>> >                                     <xs:all>
> >>> >                                         <xs:element name="id"
> >>> > type="xs:string" maxOccurs="1" minOccurs="1" nillable="false"/>
> >>> >                                         <xs:element
> name="partitionMax"
> >>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                                         <xs:element
> name="partitionMin"
> >>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                                     </xs:all>
> >>> >                                 </xs:complexType>
> >>> >                             </xs:element>
> >>> >                         </xs:sequence>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >             </xs:sequence>
> >>> >         </xs:complexType>
> >>> >     </xs:element>
> >>> >
> >>> >     <xs:element name="cartridgeDefinitionBean">
> >>> >         <xs:complexType>
> >>> >             <xs:sequence>
> >>> >                 <xs:element name="type" type="xs:string"
> minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="provider" type="xs:string"
> >>> > minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="host" type="xs:string"
> minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="displayName" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="description" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="version" type="xs:float"
> >>> > minOccurs="1"
> >>> > maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="defaultAutoscalingPolicy"
> >>> > type="xs:string"
> >>> > minOccurs="0" maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="multiTenant" type="xs:boolean"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                 <xs:element name="portMapping" minOccurs="1"
> >>> > maxOccurs="unbounded">
> >>> >                     <xs:complexType>
> >>> >                         <xs:all>
> >>> >                             <xs:element name="protocol"
> >>> > type="xs:string"
> >>> > maxOccurs="1" minOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="port" type="xs:int"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="proxyPort"
> type="xs:int"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                         </xs:all>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >                 <xs:element name="iaasProvider" minOccurs="0"
> >>> > maxOccurs="unbounded">
> >>> >                     <xs:complexType>
> >>> >                         <xs:sequence>
> >>> >                             <xs:element name="type" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="imageId"
> type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="maxInstanceLimit"
> >>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="property" minOccurs="1"
> >>> > maxOccurs="unbounded">
> >>> >                                 <xs:complexType>
> >>> >                                     <xs:all>
> >>> >                                         <xs:element name="name"
> >>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                                         <xs:element name="value"
> >>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                                     </xs:all>
> >>> >                                 </xs:complexType>
> >>> >                             </xs:element>
> >>> >                         </xs:sequence>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >                 <xs:element name="loadBalancer" minOccurs="0"
> >>> > maxOccurs="1"
> >>> > nillable="true">
> >>> >                     <xs:complexType>
> >>> >                         <xs:sequence>
> >>> >                             <xs:element name="type" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="property" minOccurs="1"
> >>> > maxOccurs="unbounded">
> >>> >                                 <xs:complexType>
> >>> >                                     <xs:all>
> >>> >                                         <xs:element name="name"
> >>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                                         <xs:element name="value"
> >>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                                     </xs:all>
> >>> >                                 </xs:complexType>
> >>> >                             </xs:element>
> >>> >                         </xs:sequence>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >                 <xs:element name="property" minOccurs="0"
> >>> > maxOccurs="unbounded">
> >>> >                     <xs:complexType>
> >>> >                         <xs:all>
> >>> >                             <xs:element name="name" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                             <xs:element name="value" type="xs:string"
> >>> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >>> >                         </xs:all>
> >>> >                     </xs:complexType>
> >>> >                 </xs:element>
> >>> >             </xs:sequence>
> >>> >         </xs:complexType>
> >>> >     </xs:element>
> >>> >
> >>> >     <xs:complexType name="thresholdValues">
> >>> >         <xs:all>
> >>> >             <xs:element name="average" type="xs:int" minOccurs="1"
> >>> > maxOccurs="1"/>
> >>> >             <xs:element name="gradient" type="xs:float" minOccurs="1"
> >>> > maxOccurs="1"/>
> >>> >             <xs:element name="secondDerivative" type="xs:float"
> >>> > minOccurs="1" maxOccurs="1"/>
> >>> >             <xs:element name="scaleDownMarginOfGradient"
> >>> > type="xs:float"
> >>> > minOccurs="1" maxOccurs="1"/>
> >>> >             <xs:element name="scaleDownMarginOfSecondDerivative"
> >>> > type="xs:float" minOccurs="1" maxOccurs="1"/>
> >>> >         </xs:all>
> >>> >     </xs:complexType>
> >>> > </xs:schema>
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Akila Ravihansa Perera
> >>> Software Engineer
> >>> WSO2 Inc.
> >>> http://wso2.com
> >>>
> >>> Phone: +94 77 64 154 38
> >>> Blog: http://ravihansa3000.blogspot.com
> >>
> >>
> >>
> >>
> >> --
> >> Pradeep Fernando.
> >> http://pradeepfernando.blogspot.com/
> >
> >
> >
> >
> > --
> > Best Regards,
> > Nirmal
> >
> > Nirmal Fernando.
> > PPMC Member & Committer of Apache Stratos,
> > Senior Software Engineer, WSO2 Inc.
> >
> > Blog: http://nirmalfdo.blogspot.com/
>
>
>
> --
> Akila Ravihansa Perera
> Software Engineer, WSO2
> Committer & PMC Member, Apache Stratos
>
> Blog: http://ravihansa3000.blogspot.com
>



-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Validating Inputs to REST API

Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi,

Bringing up this thread back since validation is broken in master
branch. I've fixed validations schema to reflect the recent changes
done to auto-scaling policy (in STRATOS-645).

@Nirmal, can you remember what policies were not covered in the
validation schema? I did some cross checking but I guess I'm missing
something here.

Thanks.

On Wed, May 7, 2014 at 4:11 AM, Nirmal Fernando <ni...@gmail.com> wrote:
> Hi Pradeep,
>
> XSD schema seems validating only few levels of the policy files. Is there
> any reason for not validating full config?
>
>
> On Tue, Apr 29, 2014 at 10:47 AM, Pradeep Fernando <pr...@gmail.com>
> wrote:
>>
>> Hi Akila,
>>
>>
>> On Mon, Apr 28, 2014 at 1:24 PM, Akila Ravihansa Perera
>> <ra...@wso2.com> wrote:
>>>
>>> Hi Pradeep,
>>>
>>> I have some concerns regarding the schema.
>>>
>>> 1. In partition deployment, do we really need to make it mandatory to
>>> include partitionMin and partitionMax?
>>> IMO, minOccurs should be 0 for these two properties since this can be
>>> overridden in the deployment definition
>>
>>
>> +1. Please go ahead and change
>>>
>>>
>>> 2. What is the need for having a sequence in properties definition?
>>> For eg - In cartridge definition if the user don't enter the values in
>>> the order that you have defined in the schema it won't validate. IMO,
>>> we should use <xs:choice> instead of  <xs:sequence> when describing
>>> properties.
>>
>>
>> Sorry i did not get you. I used xs:all when definining property arrays.
>>
>>>
>>>
>>> Thanks.
>>>
>>> On Wed, Apr 23, 2014 at 4:22 PM, Pradeep Fernando <pr...@gmail.com>
>>> wrote:
>>> > Hi devs,
>>> >
>>> > Now the API spits error messages irrespective of the backend exception.
>>> > It
>>> > sends the JSON error message even if there is an Nullpointer in our
>>> > backend.
>>> > Writing a exception mapper for throwable did the trick.
>>> >
>>> > Here is the schema I came up with for validating input. I derived it
>>> > from
>>> > the sample inputs i have. Please review it. Pretty sure there is space
>>> > for
>>> > improvements particularly min occurs.
>>> >
>>> >
>>> > <?xml version="1.0" encoding="UTF-8" ?>
>>> > <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
>>> >     <xs:element name="partition">
>>> >         <xs:complexType>
>>> >             <xs:sequence>
>>> >                 <xs:element name="id" type="xs:string" minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="provider" type="xs:string"
>>> > minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="property" minOccurs="1"
>>> > maxOccurs="unbounded">
>>> >                     <xs:complexType>
>>> >                         <xs:all>
>>> >                             <xs:element name="name" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="value" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                         </xs:all>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >                 <xs:element name="partitionMin" type="xs:int"
>>> > minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="partitionMax" type="xs:int"
>>> > minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >             </xs:sequence>
>>> >         </xs:complexType>
>>> >     </xs:element>
>>> >
>>> >     <xs:element name="autoscalePolicy">
>>> >         <xs:complexType>
>>> >             <xs:sequence>
>>> >                 <xs:element name="id" type="xs:string" minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="loadThresholds" minOccurs="1"
>>> > maxOccurs="1" nillable="false">
>>> >                     <xs:complexType>
>>> >                         <xs:all>
>>> >                             <xs:element name="requestsInFlight"
>>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="memoryConsumption"
>>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="loadAverage"
>>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                         </xs:all>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >             </xs:sequence>
>>> >         </xs:complexType>
>>> >     </xs:element>
>>> >
>>> >     <xs:element name="deploymentPolicy">
>>> >         <xs:complexType>
>>> >             <xs:sequence>
>>> >                 <xs:element name="id" type="xs:string" maxOccurs="1"
>>> > minOccurs="1" nillable="false"/>
>>> >                 <xs:element name="partitionGroup">
>>> >                     <xs:complexType>
>>> >                         <xs:sequence>
>>> >                             <xs:element name="id" type="xs:string"
>>> > maxOccurs="1" minOccurs="1" nillable="false"/>
>>> >                             <xs:element name="partitionAlgo"
>>> > minOccurs="1"
>>> > maxOccurs="1" nillable="false">
>>> >                                 <xs:simpleType>
>>> >                                     <xs:restriction base="xs:string">
>>> >                                         <xs:enumeration
>>> > value="round-robin"/>
>>> >                                         <xs:enumeration
>>> > value="one-after-another"/>
>>> >                                     </xs:restriction>
>>> >                                 </xs:simpleType>
>>> >                             </xs:element>
>>> >                             <xs:element name="partition" minOccurs="1"
>>> > maxOccurs="unbounded">
>>> >                                 <xs:complexType>
>>> >                                     <xs:all>
>>> >                                         <xs:element name="id"
>>> > type="xs:string" maxOccurs="1" minOccurs="1" nillable="false"/>
>>> >                                         <xs:element name="partitionMax"
>>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                                         <xs:element name="partitionMin"
>>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                                     </xs:all>
>>> >                                 </xs:complexType>
>>> >                             </xs:element>
>>> >                         </xs:sequence>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >             </xs:sequence>
>>> >         </xs:complexType>
>>> >     </xs:element>
>>> >
>>> >     <xs:element name="cartridgeDefinitionBean">
>>> >         <xs:complexType>
>>> >             <xs:sequence>
>>> >                 <xs:element name="type" type="xs:string" minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="provider" type="xs:string"
>>> > minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="host" type="xs:string" minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="displayName" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="description" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="version" type="xs:float"
>>> > minOccurs="1"
>>> > maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="defaultAutoscalingPolicy"
>>> > type="xs:string"
>>> > minOccurs="0" maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="multiTenant" type="xs:boolean"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                 <xs:element name="portMapping" minOccurs="1"
>>> > maxOccurs="unbounded">
>>> >                     <xs:complexType>
>>> >                         <xs:all>
>>> >                             <xs:element name="protocol"
>>> > type="xs:string"
>>> > maxOccurs="1" minOccurs="1" nillable="false"/>
>>> >                             <xs:element name="port" type="xs:int"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="proxyPort" type="xs:int"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                         </xs:all>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >                 <xs:element name="iaasProvider" minOccurs="0"
>>> > maxOccurs="unbounded">
>>> >                     <xs:complexType>
>>> >                         <xs:sequence>
>>> >                             <xs:element name="type" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="imageId" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="maxInstanceLimit"
>>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="property" minOccurs="1"
>>> > maxOccurs="unbounded">
>>> >                                 <xs:complexType>
>>> >                                     <xs:all>
>>> >                                         <xs:element name="name"
>>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                                         <xs:element name="value"
>>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                                     </xs:all>
>>> >                                 </xs:complexType>
>>> >                             </xs:element>
>>> >                         </xs:sequence>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >                 <xs:element name="loadBalancer" minOccurs="0"
>>> > maxOccurs="1"
>>> > nillable="true">
>>> >                     <xs:complexType>
>>> >                         <xs:sequence>
>>> >                             <xs:element name="type" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="property" minOccurs="1"
>>> > maxOccurs="unbounded">
>>> >                                 <xs:complexType>
>>> >                                     <xs:all>
>>> >                                         <xs:element name="name"
>>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                                         <xs:element name="value"
>>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                                     </xs:all>
>>> >                                 </xs:complexType>
>>> >                             </xs:element>
>>> >                         </xs:sequence>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >                 <xs:element name="property" minOccurs="0"
>>> > maxOccurs="unbounded">
>>> >                     <xs:complexType>
>>> >                         <xs:all>
>>> >                             <xs:element name="name" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                             <xs:element name="value" type="xs:string"
>>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>>> >                         </xs:all>
>>> >                     </xs:complexType>
>>> >                 </xs:element>
>>> >             </xs:sequence>
>>> >         </xs:complexType>
>>> >     </xs:element>
>>> >
>>> >     <xs:complexType name="thresholdValues">
>>> >         <xs:all>
>>> >             <xs:element name="average" type="xs:int" minOccurs="1"
>>> > maxOccurs="1"/>
>>> >             <xs:element name="gradient" type="xs:float" minOccurs="1"
>>> > maxOccurs="1"/>
>>> >             <xs:element name="secondDerivative" type="xs:float"
>>> > minOccurs="1" maxOccurs="1"/>
>>> >             <xs:element name="scaleDownMarginOfGradient"
>>> > type="xs:float"
>>> > minOccurs="1" maxOccurs="1"/>
>>> >             <xs:element name="scaleDownMarginOfSecondDerivative"
>>> > type="xs:float" minOccurs="1" maxOccurs="1"/>
>>> >         </xs:all>
>>> >     </xs:complexType>
>>> > </xs:schema>
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Akila Ravihansa Perera
>>> Software Engineer
>>> WSO2 Inc.
>>> http://wso2.com
>>>
>>> Phone: +94 77 64 154 38
>>> Blog: http://ravihansa3000.blogspot.com
>>
>>
>>
>>
>> --
>> Pradeep Fernando.
>> http://pradeepfernando.blogspot.com/
>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/



-- 
Akila Ravihansa Perera
Software Engineer, WSO2
Committer & PMC Member, Apache Stratos

Blog: http://ravihansa3000.blogspot.com

Re: Validating Inputs to REST API

Posted by Nirmal Fernando <ni...@gmail.com>.
Hi Pradeep,

XSD schema seems validating only few levels of the policy files. Is there
any reason for not validating full config?


On Tue, Apr 29, 2014 at 10:47 AM, Pradeep Fernando <pr...@gmail.com>wrote:

> Hi Akila,
>
>
> On Mon, Apr 28, 2014 at 1:24 PM, Akila Ravihansa Perera <
> ravihansa@wso2.com> wrote:
>
>> Hi Pradeep,
>>
>> I have some concerns regarding the schema.
>>
>> 1. In partition deployment, do we really need to make it mandatory to
>> include partitionMin and partitionMax?
>> IMO, minOccurs should be 0 for these two properties since this can be
>> overridden in the deployment definition
>>
>
> +1. Please go ahead and change
>
>>
>> 2. What is the need for having a sequence in properties definition?
>> For eg - In cartridge definition if the user don't enter the values in
>> the order that you have defined in the schema it won't validate. IMO,
>> we should use <xs:choice> instead of  <xs:sequence> when describing
>> properties.
>>
>
> Sorry i did not get you. I used xs:all when definining property arrays.
>
>
>>
>> Thanks.
>>
>> On Wed, Apr 23, 2014 at 4:22 PM, Pradeep Fernando <pr...@gmail.com>
>> wrote:
>> > Hi devs,
>> >
>> > Now the API spits error messages irrespective of the backend exception.
>> It
>> > sends the JSON error message even if there is an Nullpointer in our
>> backend.
>> > Writing a exception mapper for throwable did the trick.
>> >
>> > Here is the schema I came up with for validating input. I derived it
>> from
>> > the sample inputs i have. Please review it. Pretty sure there is space
>> for
>> > improvements particularly min occurs.
>> >
>> >
>> > <?xml version="1.0" encoding="UTF-8" ?>
>> > <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
>> >     <xs:element name="partition">
>> >         <xs:complexType>
>> >             <xs:sequence>
>> >                 <xs:element name="id" type="xs:string" minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="provider" type="xs:string"
>> minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="property" minOccurs="1"
>> > maxOccurs="unbounded">
>> >                     <xs:complexType>
>> >                         <xs:all>
>> >                             <xs:element name="name" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="value" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                         </xs:all>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >                 <xs:element name="partitionMin" type="xs:int"
>> minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="partitionMax" type="xs:int"
>> minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >             </xs:sequence>
>> >         </xs:complexType>
>> >     </xs:element>
>> >
>> >     <xs:element name="autoscalePolicy">
>> >         <xs:complexType>
>> >             <xs:sequence>
>> >                 <xs:element name="id" type="xs:string" minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="loadThresholds" minOccurs="1"
>> > maxOccurs="1" nillable="false">
>> >                     <xs:complexType>
>> >                         <xs:all>
>> >                             <xs:element name="requestsInFlight"
>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="memoryConsumption"
>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="loadAverage"
>> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                         </xs:all>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >             </xs:sequence>
>> >         </xs:complexType>
>> >     </xs:element>
>> >
>> >     <xs:element name="deploymentPolicy">
>> >         <xs:complexType>
>> >             <xs:sequence>
>> >                 <xs:element name="id" type="xs:string" maxOccurs="1"
>> > minOccurs="1" nillable="false"/>
>> >                 <xs:element name="partitionGroup">
>> >                     <xs:complexType>
>> >                         <xs:sequence>
>> >                             <xs:element name="id" type="xs:string"
>> > maxOccurs="1" minOccurs="1" nillable="false"/>
>> >                             <xs:element name="partitionAlgo"
>> minOccurs="1"
>> > maxOccurs="1" nillable="false">
>> >                                 <xs:simpleType>
>> >                                     <xs:restriction base="xs:string">
>> >                                         <xs:enumeration
>> > value="round-robin"/>
>> >                                         <xs:enumeration
>> > value="one-after-another"/>
>> >                                     </xs:restriction>
>> >                                 </xs:simpleType>
>> >                             </xs:element>
>> >                             <xs:element name="partition" minOccurs="1"
>> > maxOccurs="unbounded">
>> >                                 <xs:complexType>
>> >                                     <xs:all>
>> >                                         <xs:element name="id"
>> > type="xs:string" maxOccurs="1" minOccurs="1" nillable="false"/>
>> >                                         <xs:element name="partitionMax"
>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                                         <xs:element name="partitionMin"
>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                                     </xs:all>
>> >                                 </xs:complexType>
>> >                             </xs:element>
>> >                         </xs:sequence>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >             </xs:sequence>
>> >         </xs:complexType>
>> >     </xs:element>
>> >
>> >     <xs:element name="cartridgeDefinitionBean">
>> >         <xs:complexType>
>> >             <xs:sequence>
>> >                 <xs:element name="type" type="xs:string" minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="provider" type="xs:string"
>> minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="host" type="xs:string" minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="displayName" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="description" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="version" type="xs:float" minOccurs="1"
>> > maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="defaultAutoscalingPolicy"
>> type="xs:string"
>> > minOccurs="0" maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="multiTenant" type="xs:boolean"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                 <xs:element name="portMapping" minOccurs="1"
>> > maxOccurs="unbounded">
>> >                     <xs:complexType>
>> >                         <xs:all>
>> >                             <xs:element name="protocol" type="xs:string"
>> > maxOccurs="1" minOccurs="1" nillable="false"/>
>> >                             <xs:element name="port" type="xs:int"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="proxyPort" type="xs:int"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                         </xs:all>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >                 <xs:element name="iaasProvider" minOccurs="0"
>> > maxOccurs="unbounded">
>> >                     <xs:complexType>
>> >                         <xs:sequence>
>> >                             <xs:element name="type" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="imageId" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="maxInstanceLimit"
>> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="property" minOccurs="1"
>> > maxOccurs="unbounded">
>> >                                 <xs:complexType>
>> >                                     <xs:all>
>> >                                         <xs:element name="name"
>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                                         <xs:element name="value"
>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                                     </xs:all>
>> >                                 </xs:complexType>
>> >                             </xs:element>
>> >                         </xs:sequence>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >                 <xs:element name="loadBalancer" minOccurs="0"
>> maxOccurs="1"
>> > nillable="true">
>> >                     <xs:complexType>
>> >                         <xs:sequence>
>> >                             <xs:element name="type" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="property" minOccurs="1"
>> > maxOccurs="unbounded">
>> >                                 <xs:complexType>
>> >                                     <xs:all>
>> >                                         <xs:element name="name"
>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                                         <xs:element name="value"
>> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                                     </xs:all>
>> >                                 </xs:complexType>
>> >                             </xs:element>
>> >                         </xs:sequence>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >                 <xs:element name="property" minOccurs="0"
>> > maxOccurs="unbounded">
>> >                     <xs:complexType>
>> >                         <xs:all>
>> >                             <xs:element name="name" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                             <xs:element name="value" type="xs:string"
>> > minOccurs="1" maxOccurs="1" nillable="false"/>
>> >                         </xs:all>
>> >                     </xs:complexType>
>> >                 </xs:element>
>> >             </xs:sequence>
>> >         </xs:complexType>
>> >     </xs:element>
>> >
>> >     <xs:complexType name="thresholdValues">
>> >         <xs:all>
>> >             <xs:element name="average" type="xs:int" minOccurs="1"
>> > maxOccurs="1"/>
>> >             <xs:element name="gradient" type="xs:float" minOccurs="1"
>> > maxOccurs="1"/>
>> >             <xs:element name="secondDerivative" type="xs:float"
>> > minOccurs="1" maxOccurs="1"/>
>> >             <xs:element name="scaleDownMarginOfGradient" type="xs:float"
>> > minOccurs="1" maxOccurs="1"/>
>> >             <xs:element name="scaleDownMarginOfSecondDerivative"
>> > type="xs:float" minOccurs="1" maxOccurs="1"/>
>> >         </xs:all>
>> >     </xs:complexType>
>> > </xs:schema>
>> >
>> >
>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> Software Engineer
>> WSO2 Inc.
>> http://wso2.com
>>
>> Phone: +94 77 64 154 38
>> Blog: http://ravihansa3000.blogspot.com
>>
>
>
>
> --
> Pradeep Fernando.
> http://pradeepfernando.blogspot.com/
>



-- 
Best Regards,
Nirmal

Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.

Blog: http://nirmalfdo.blogspot.com/

Re: Validating Inputs to REST API

Posted by Pradeep Fernando <pr...@gmail.com>.
Hi Akila,


On Mon, Apr 28, 2014 at 1:24 PM, Akila Ravihansa Perera
<ra...@wso2.com>wrote:

> Hi Pradeep,
>
> I have some concerns regarding the schema.
>
> 1. In partition deployment, do we really need to make it mandatory to
> include partitionMin and partitionMax?
> IMO, minOccurs should be 0 for these two properties since this can be
> overridden in the deployment definition
>

+1. Please go ahead and change

>
> 2. What is the need for having a sequence in properties definition?
> For eg - In cartridge definition if the user don't enter the values in
> the order that you have defined in the schema it won't validate. IMO,
> we should use <xs:choice> instead of  <xs:sequence> when describing
> properties.
>

Sorry i did not get you. I used xs:all when definining property arrays.


>
> Thanks.
>
> On Wed, Apr 23, 2014 at 4:22 PM, Pradeep Fernando <pr...@gmail.com>
> wrote:
> > Hi devs,
> >
> > Now the API spits error messages irrespective of the backend exception.
> It
> > sends the JSON error message even if there is an Nullpointer in our
> backend.
> > Writing a exception mapper for throwable did the trick.
> >
> > Here is the schema I came up with for validating input. I derived it from
> > the sample inputs i have. Please review it. Pretty sure there is space
> for
> > improvements particularly min occurs.
> >
> >
> > <?xml version="1.0" encoding="UTF-8" ?>
> > <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
> >     <xs:element name="partition">
> >         <xs:complexType>
> >             <xs:sequence>
> >                 <xs:element name="id" type="xs:string" minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="provider" type="xs:string"
> minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="property" minOccurs="1"
> > maxOccurs="unbounded">
> >                     <xs:complexType>
> >                         <xs:all>
> >                             <xs:element name="name" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="value" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                         </xs:all>
> >                     </xs:complexType>
> >                 </xs:element>
> >                 <xs:element name="partitionMin" type="xs:int"
> minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="partitionMax" type="xs:int"
> minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >             </xs:sequence>
> >         </xs:complexType>
> >     </xs:element>
> >
> >     <xs:element name="autoscalePolicy">
> >         <xs:complexType>
> >             <xs:sequence>
> >                 <xs:element name="id" type="xs:string" minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="loadThresholds" minOccurs="1"
> > maxOccurs="1" nillable="false">
> >                     <xs:complexType>
> >                         <xs:all>
> >                             <xs:element name="requestsInFlight"
> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="memoryConsumption"
> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="loadAverage"
> > type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                         </xs:all>
> >                     </xs:complexType>
> >                 </xs:element>
> >             </xs:sequence>
> >         </xs:complexType>
> >     </xs:element>
> >
> >     <xs:element name="deploymentPolicy">
> >         <xs:complexType>
> >             <xs:sequence>
> >                 <xs:element name="id" type="xs:string" maxOccurs="1"
> > minOccurs="1" nillable="false"/>
> >                 <xs:element name="partitionGroup">
> >                     <xs:complexType>
> >                         <xs:sequence>
> >                             <xs:element name="id" type="xs:string"
> > maxOccurs="1" minOccurs="1" nillable="false"/>
> >                             <xs:element name="partitionAlgo"
> minOccurs="1"
> > maxOccurs="1" nillable="false">
> >                                 <xs:simpleType>
> >                                     <xs:restriction base="xs:string">
> >                                         <xs:enumeration
> > value="round-robin"/>
> >                                         <xs:enumeration
> > value="one-after-another"/>
> >                                     </xs:restriction>
> >                                 </xs:simpleType>
> >                             </xs:element>
> >                             <xs:element name="partition" minOccurs="1"
> > maxOccurs="unbounded">
> >                                 <xs:complexType>
> >                                     <xs:all>
> >                                         <xs:element name="id"
> > type="xs:string" maxOccurs="1" minOccurs="1" nillable="false"/>
> >                                         <xs:element name="partitionMax"
> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                                         <xs:element name="partitionMin"
> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                                     </xs:all>
> >                                 </xs:complexType>
> >                             </xs:element>
> >                         </xs:sequence>
> >                     </xs:complexType>
> >                 </xs:element>
> >             </xs:sequence>
> >         </xs:complexType>
> >     </xs:element>
> >
> >     <xs:element name="cartridgeDefinitionBean">
> >         <xs:complexType>
> >             <xs:sequence>
> >                 <xs:element name="type" type="xs:string" minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="provider" type="xs:string"
> minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="host" type="xs:string" minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="displayName" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                 <xs:element name="description" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                 <xs:element name="version" type="xs:float" minOccurs="1"
> > maxOccurs="1" nillable="false"/>
> >                 <xs:element name="defaultAutoscalingPolicy"
> type="xs:string"
> > minOccurs="0" maxOccurs="1" nillable="false"/>
> >                 <xs:element name="multiTenant" type="xs:boolean"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                 <xs:element name="portMapping" minOccurs="1"
> > maxOccurs="unbounded">
> >                     <xs:complexType>
> >                         <xs:all>
> >                             <xs:element name="protocol" type="xs:string"
> > maxOccurs="1" minOccurs="1" nillable="false"/>
> >                             <xs:element name="port" type="xs:int"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="proxyPort" type="xs:int"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                         </xs:all>
> >                     </xs:complexType>
> >                 </xs:element>
> >                 <xs:element name="iaasProvider" minOccurs="0"
> > maxOccurs="unbounded">
> >                     <xs:complexType>
> >                         <xs:sequence>
> >                             <xs:element name="type" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="imageId" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="maxInstanceLimit"
> > type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="property" minOccurs="1"
> > maxOccurs="unbounded">
> >                                 <xs:complexType>
> >                                     <xs:all>
> >                                         <xs:element name="name"
> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                                         <xs:element name="value"
> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                                     </xs:all>
> >                                 </xs:complexType>
> >                             </xs:element>
> >                         </xs:sequence>
> >                     </xs:complexType>
> >                 </xs:element>
> >                 <xs:element name="loadBalancer" minOccurs="0"
> maxOccurs="1"
> > nillable="true">
> >                     <xs:complexType>
> >                         <xs:sequence>
> >                             <xs:element name="type" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="property" minOccurs="1"
> > maxOccurs="unbounded">
> >                                 <xs:complexType>
> >                                     <xs:all>
> >                                         <xs:element name="name"
> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                                         <xs:element name="value"
> > type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
> >                                     </xs:all>
> >                                 </xs:complexType>
> >                             </xs:element>
> >                         </xs:sequence>
> >                     </xs:complexType>
> >                 </xs:element>
> >                 <xs:element name="property" minOccurs="0"
> > maxOccurs="unbounded">
> >                     <xs:complexType>
> >                         <xs:all>
> >                             <xs:element name="name" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                             <xs:element name="value" type="xs:string"
> > minOccurs="1" maxOccurs="1" nillable="false"/>
> >                         </xs:all>
> >                     </xs:complexType>
> >                 </xs:element>
> >             </xs:sequence>
> >         </xs:complexType>
> >     </xs:element>
> >
> >     <xs:complexType name="thresholdValues">
> >         <xs:all>
> >             <xs:element name="average" type="xs:int" minOccurs="1"
> > maxOccurs="1"/>
> >             <xs:element name="gradient" type="xs:float" minOccurs="1"
> > maxOccurs="1"/>
> >             <xs:element name="secondDerivative" type="xs:float"
> > minOccurs="1" maxOccurs="1"/>
> >             <xs:element name="scaleDownMarginOfGradient" type="xs:float"
> > minOccurs="1" maxOccurs="1"/>
> >             <xs:element name="scaleDownMarginOfSecondDerivative"
> > type="xs:float" minOccurs="1" maxOccurs="1"/>
> >         </xs:all>
> >     </xs:complexType>
> > </xs:schema>
> >
> >
>
>
>
> --
> Akila Ravihansa Perera
> Software Engineer
> WSO2 Inc.
> http://wso2.com
>
> Phone: +94 77 64 154 38
> Blog: http://ravihansa3000.blogspot.com
>



-- 
Pradeep Fernando.
http://pradeepfernando.blogspot.com/

Re: Validating Inputs to REST API

Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi Pradeep,

I have some concerns regarding the schema.

1. In partition deployment, do we really need to make it mandatory to
include partitionMin and partitionMax?
IMO, minOccurs should be 0 for these two properties since this can be
overridden in the deployment definition

2. What is the need for having a sequence in properties definition?
For eg - In cartridge definition if the user don't enter the values in
the order that you have defined in the schema it won't validate. IMO,
we should use <xs:choice> instead of  <xs:sequence> when describing
properties.

Thanks.

On Wed, Apr 23, 2014 at 4:22 PM, Pradeep Fernando <pr...@gmail.com> wrote:
> Hi devs,
>
> Now the API spits error messages irrespective of the backend exception. It
> sends the JSON error message even if there is an Nullpointer in our backend.
> Writing a exception mapper for throwable did the trick.
>
> Here is the schema I came up with for validating input. I derived it from
> the sample inputs i have. Please review it. Pretty sure there is space for
> improvements particularly min occurs.
>
>
> <?xml version="1.0" encoding="UTF-8" ?>
> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
>     <xs:element name="partition">
>         <xs:complexType>
>             <xs:sequence>
>                 <xs:element name="id" type="xs:string" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="provider" type="xs:string" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="property" minOccurs="1"
> maxOccurs="unbounded">
>                     <xs:complexType>
>                         <xs:all>
>                             <xs:element name="name" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="value" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                         </xs:all>
>                     </xs:complexType>
>                 </xs:element>
>                 <xs:element name="partitionMin" type="xs:int" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="partitionMax" type="xs:int" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>             </xs:sequence>
>         </xs:complexType>
>     </xs:element>
>
>     <xs:element name="autoscalePolicy">
>         <xs:complexType>
>             <xs:sequence>
>                 <xs:element name="id" type="xs:string" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="loadThresholds" minOccurs="1"
> maxOccurs="1" nillable="false">
>                     <xs:complexType>
>                         <xs:all>
>                             <xs:element name="requestsInFlight"
> type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="memoryConsumption"
> type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="loadAverage"
> type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
>                         </xs:all>
>                     </xs:complexType>
>                 </xs:element>
>             </xs:sequence>
>         </xs:complexType>
>     </xs:element>
>
>     <xs:element name="deploymentPolicy">
>         <xs:complexType>
>             <xs:sequence>
>                 <xs:element name="id" type="xs:string" maxOccurs="1"
> minOccurs="1" nillable="false"/>
>                 <xs:element name="partitionGroup">
>                     <xs:complexType>
>                         <xs:sequence>
>                             <xs:element name="id" type="xs:string"
> maxOccurs="1" minOccurs="1" nillable="false"/>
>                             <xs:element name="partitionAlgo" minOccurs="1"
> maxOccurs="1" nillable="false">
>                                 <xs:simpleType>
>                                     <xs:restriction base="xs:string">
>                                         <xs:enumeration
> value="round-robin"/>
>                                         <xs:enumeration
> value="one-after-another"/>
>                                     </xs:restriction>
>                                 </xs:simpleType>
>                             </xs:element>
>                             <xs:element name="partition" minOccurs="1"
> maxOccurs="unbounded">
>                                 <xs:complexType>
>                                     <xs:all>
>                                         <xs:element name="id"
> type="xs:string" maxOccurs="1" minOccurs="1" nillable="false"/>
>                                         <xs:element name="partitionMax"
> type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>                                         <xs:element name="partitionMin"
> type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>                                     </xs:all>
>                                 </xs:complexType>
>                             </xs:element>
>                         </xs:sequence>
>                     </xs:complexType>
>                 </xs:element>
>             </xs:sequence>
>         </xs:complexType>
>     </xs:element>
>
>     <xs:element name="cartridgeDefinitionBean">
>         <xs:complexType>
>             <xs:sequence>
>                 <xs:element name="type" type="xs:string" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="provider" type="xs:string" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="host" type="xs:string" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="displayName" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                 <xs:element name="description" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                 <xs:element name="version" type="xs:float" minOccurs="1"
> maxOccurs="1" nillable="false"/>
>                 <xs:element name="defaultAutoscalingPolicy" type="xs:string"
> minOccurs="0" maxOccurs="1" nillable="false"/>
>                 <xs:element name="multiTenant" type="xs:boolean"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                 <xs:element name="portMapping" minOccurs="1"
> maxOccurs="unbounded">
>                     <xs:complexType>
>                         <xs:all>
>                             <xs:element name="protocol" type="xs:string"
> maxOccurs="1" minOccurs="1" nillable="false"/>
>                             <xs:element name="port" type="xs:int"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="proxyPort" type="xs:int"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                         </xs:all>
>                     </xs:complexType>
>                 </xs:element>
>                 <xs:element name="iaasProvider" minOccurs="0"
> maxOccurs="unbounded">
>                     <xs:complexType>
>                         <xs:sequence>
>                             <xs:element name="type" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="imageId" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="maxInstanceLimit"
> type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="property" minOccurs="1"
> maxOccurs="unbounded">
>                                 <xs:complexType>
>                                     <xs:all>
>                                         <xs:element name="name"
> type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>                                         <xs:element name="value"
> type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>                                     </xs:all>
>                                 </xs:complexType>
>                             </xs:element>
>                         </xs:sequence>
>                     </xs:complexType>
>                 </xs:element>
>                 <xs:element name="loadBalancer" minOccurs="0" maxOccurs="1"
> nillable="true">
>                     <xs:complexType>
>                         <xs:sequence>
>                             <xs:element name="type" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="property" minOccurs="1"
> maxOccurs="unbounded">
>                                 <xs:complexType>
>                                     <xs:all>
>                                         <xs:element name="name"
> type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>                                         <xs:element name="value"
> type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
>                                     </xs:all>
>                                 </xs:complexType>
>                             </xs:element>
>                         </xs:sequence>
>                     </xs:complexType>
>                 </xs:element>
>                 <xs:element name="property" minOccurs="0"
> maxOccurs="unbounded">
>                     <xs:complexType>
>                         <xs:all>
>                             <xs:element name="name" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                             <xs:element name="value" type="xs:string"
> minOccurs="1" maxOccurs="1" nillable="false"/>
>                         </xs:all>
>                     </xs:complexType>
>                 </xs:element>
>             </xs:sequence>
>         </xs:complexType>
>     </xs:element>
>
>     <xs:complexType name="thresholdValues">
>         <xs:all>
>             <xs:element name="average" type="xs:int" minOccurs="1"
> maxOccurs="1"/>
>             <xs:element name="gradient" type="xs:float" minOccurs="1"
> maxOccurs="1"/>
>             <xs:element name="secondDerivative" type="xs:float"
> minOccurs="1" maxOccurs="1"/>
>             <xs:element name="scaleDownMarginOfGradient" type="xs:float"
> minOccurs="1" maxOccurs="1"/>
>             <xs:element name="scaleDownMarginOfSecondDerivative"
> type="xs:float" minOccurs="1" maxOccurs="1"/>
>         </xs:all>
>     </xs:complexType>
> </xs:schema>
>
>



-- 
Akila Ravihansa Perera
Software Engineer
WSO2 Inc.
http://wso2.com

Phone: +94 77 64 154 38
Blog: http://ravihansa3000.blogspot.com

Re: Validating Inputs to REST API

Posted by Pradeep Fernando <pr...@gmail.com>.
Hi devs,

Now the API spits error messages irrespective of the backend exception. It
sends the JSON error message even if there is an Nullpointer in our
backend. Writing a exception mapper for throwable did the trick.

Here is the schema I came up with for validating input. I derived it from
the sample inputs i have. Please review it. Pretty sure there is space for
improvements particularly min occurs.


<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
    <xs:element name="partition">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="id" type="xs:string" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="provider" type="xs:string" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="property" minOccurs="1"
maxOccurs="unbounded">
                    <xs:complexType>
                        <xs:all>
                            <xs:element name="name" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="value" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                        </xs:all>
                    </xs:complexType>
                </xs:element>
                <xs:element name="partitionMin" type="xs:int" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="partitionMax" type="xs:int" minOccurs="1"
maxOccurs="1" nillable="false"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>

    <xs:element name="autoscalePolicy">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="id" type="xs:string" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="loadThresholds" minOccurs="1"
maxOccurs="1" nillable="false">
                    <xs:complexType>
                        <xs:all>
                            <xs:element name="requestsInFlight"
type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="memoryConsumption"
type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="loadAverage"
type="thresholdValues" minOccurs="1" maxOccurs="1" nillable="false"/>
                        </xs:all>
                    </xs:complexType>
                </xs:element>
            </xs:sequence>
        </xs:complexType>
    </xs:element>

    <xs:element name="deploymentPolicy">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="id" type="xs:string" maxOccurs="1"
minOccurs="1" nillable="false"/>
                <xs:element name="partitionGroup">
                    <xs:complexType>
                        <xs:sequence>
                            <xs:element name="id" type="xs:string"
maxOccurs="1" minOccurs="1" nillable="false"/>
                            <xs:element name="partitionAlgo" minOccurs="1"
maxOccurs="1" nillable="false">
                                <xs:simpleType>
                                    <xs:restriction base="xs:string">
                                        <xs:enumeration
value="round-robin"/>
                                        <xs:enumeration
value="one-after-another"/>
                                    </xs:restriction>
                                </xs:simpleType>
                            </xs:element>
                            <xs:element name="partition" minOccurs="1"
maxOccurs="unbounded">
                                <xs:complexType>
                                    <xs:all>
                                        <xs:element name="id"
type="xs:string" maxOccurs="1" minOccurs="1" nillable="false"/>
                                        <xs:element name="partitionMax"
type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
                                        <xs:element name="partitionMin"
type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
                                    </xs:all>
                                </xs:complexType>
                            </xs:element>
                        </xs:sequence>
                    </xs:complexType>
                </xs:element>
            </xs:sequence>
        </xs:complexType>
    </xs:element>

    <xs:element name="cartridgeDefinitionBean">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="type" type="xs:string" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="provider" type="xs:string" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="host" type="xs:string" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="displayName" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                <xs:element name="description" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                <xs:element name="version" type="xs:float" minOccurs="1"
maxOccurs="1" nillable="false"/>
                <xs:element name="defaultAutoscalingPolicy"
type="xs:string" minOccurs="0" maxOccurs="1" nillable="false"/>
                <xs:element name="multiTenant" type="xs:boolean"
minOccurs="1" maxOccurs="1" nillable="false"/>
                <xs:element name="portMapping" minOccurs="1"
maxOccurs="unbounded">
                    <xs:complexType>
                        <xs:all>
                            <xs:element name="protocol" type="xs:string"
maxOccurs="1" minOccurs="1" nillable="false"/>
                            <xs:element name="port" type="xs:int"
minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="proxyPort" type="xs:int"
minOccurs="1" maxOccurs="1" nillable="false"/>
                        </xs:all>
                    </xs:complexType>
                </xs:element>
                <xs:element name="iaasProvider" minOccurs="0"
maxOccurs="unbounded">
                    <xs:complexType>
                        <xs:sequence>
                            <xs:element name="type" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="imageId" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="maxInstanceLimit"
type="xs:int" minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="property" minOccurs="1"
maxOccurs="unbounded">
                                <xs:complexType>
                                    <xs:all>
                                        <xs:element name="name"
type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
                                        <xs:element name="value"
type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
                                    </xs:all>
                                </xs:complexType>
                            </xs:element>
                        </xs:sequence>
                    </xs:complexType>
                </xs:element>
                <xs:element name="loadBalancer" minOccurs="0" maxOccurs="1"
nillable="true">
                    <xs:complexType>
                        <xs:sequence>
                            <xs:element name="type" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="property" minOccurs="1"
maxOccurs="unbounded">
                                <xs:complexType>
                                    <xs:all>
                                        <xs:element name="name"
type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
                                        <xs:element name="value"
type="xs:string" minOccurs="1" maxOccurs="1" nillable="false"/>
                                    </xs:all>
                                </xs:complexType>
                            </xs:element>
                        </xs:sequence>
                    </xs:complexType>
                </xs:element>
                <xs:element name="property" minOccurs="0"
maxOccurs="unbounded">
                    <xs:complexType>
                        <xs:all>
                            <xs:element name="name" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                            <xs:element name="value" type="xs:string"
minOccurs="1" maxOccurs="1" nillable="false"/>
                        </xs:all>
                    </xs:complexType>
                </xs:element>
            </xs:sequence>
        </xs:complexType>
    </xs:element>

    <xs:complexType name="thresholdValues">
        <xs:all>
            <xs:element name="average" type="xs:int" minOccurs="1"
maxOccurs="1"/>
            <xs:element name="gradient" type="xs:float" minOccurs="1"
maxOccurs="1"/>
            <xs:element name="secondDerivative" type="xs:float"
minOccurs="1" maxOccurs="1"/>
            <xs:element name="scaleDownMarginOfGradient" type="xs:float"
minOccurs="1" maxOccurs="1"/>
            <xs:element name="scaleDownMarginOfSecondDerivative"
type="xs:float" minOccurs="1" maxOccurs="1"/>
        </xs:all>
    </xs:complexType>
</xs:schema>