You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-dev@axis.apache.org by Shameera Rathnayaka <sh...@gmail.com> on 2012/08/08 08:47:13 UTC
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
HI devs,
I finished the implementation and did a performance test with existing SOAP
and implemented JSON. Here is the summary of the test results.
For this i used same service and send the same request in both SOAP and
JSON ways, size of JSON message is lower than SOAP message but requests are
exactly same.
Environment :
OS - Ubuntu 11.10
Axis2 1.7(current trunk)
Tool - Java benchmark
*num of requests* : 50000
*concurrency level*: 10 (Total num of requests are 50000*10 = 500000 )
*Time taken for test* : with SOAP *178* sec and with JSON *140* sec
*Total transferred* : with SOAP *2371000000* bytes and with JSON
*674000000*bytes
*Requests per second*: with SOAP * 2,801.76* [#/sec] (mean) and with
JSON *3,568.89
* [#/sec] (mean)
will do a performance test with concurrency level 50 and num of requests
500000 and give the summary of results.
Thanks,
Shameera.
On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka <
shameerainfo@gmail.com> wrote:
> HI devs,
>
> I have implemented XMLStreamWriter as well as XMLStreamReader to convert
> XML <----> JSON with XMLSchema,
> Basically it works fine, but still need few test to be done after that I
> am willing to write performance test with implemented JSON support with
> existing SOAP support.
>
> Thanks,
> Shameera.
>
>
>
> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi <
> amilasuriarachchi@gmail.com> wrote:
>
>>
>>
>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka <
>> shameerainfo@gmail.com> wrote:
>>
>>> Hi devs,
>>>
>>> I implemented this feature completely to convert XML to JSON with the
>>> help of XmlSchema of that XML document. I tested this with different
>>> Schemas, it is working fine. I implemented this as a separate project, so i
>>> will integrate this with axis2 trunk and try with actual scenario.
>>>
>>
>> Can you use this technique at the reader level as well to support
>> namespaces?
>>
>> Then do a performance comparison using a POJO service with the exiting
>> json support and based the new gson based XMLstream API. According to the
>> result we need to optimise the later to have better performance.
>>
>> thanks,
>> Amila.
>>
>>
>>>
>>> Yes, This may be a standard way to convert XML to JSON conversion as
>>> other implementations don't convert JSON message correctly when it has one
>>> value array, unless implementation supports special XML structure where
>>> Json-lib does.
>>>
>>> Definitely we can improve the performance of this implementation. May be
>>> in the way that Sagara had suggested in his reply or in another way.
>>>
>>> Thanks,
>>> Shameera.
>>>
>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi <
>>> amilasuriarachchi@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga <
>>>> sagara.gunathunga@gmail.com> wrote:
>>>>
>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
>>>>> <am...@gmail.com> wrote:
>>>>> >
>>>>> >
>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
>>>>> > <sa...@gmail.com> wrote:
>>>>> >>
>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
>>>>> >> <sh...@gmail.com> wrote:
>>>>> >> > Hi devs,
>>>>> >> >
>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I have
>>>>> >> > implemented
>>>>> >> > XMLStreamWriter API to use schema of the processing xml and gson
>>>>> >> > together
>>>>> >> > to convert XML ---> JSON. I have made my progress up to support
>>>>> xml
>>>>> >> > which
>>>>> >> > don't have complex type elements with maxOccur >1, but may have
>>>>> simple
>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point of
>>>>> view,
>>>>> >> > my
>>>>> >> > present implementation supports XML --> JSON where there is not
>>>>> any JSON
>>>>> >> > Arrays which has JSON objects in the converted JSON String.
>>>>> >> >
>>>>> >> > Here is a summary of the implementation.
>>>>> >> >
>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
>>>>> constructor
>>>>> >> > to get JsonWriter and XmlSchema object. Then it process XmlSchema
>>>>> and
>>>>> >>
>>>>> >> Are you plan to process XmlSchema object with every response ? If
>>>>> so
>>>>> >> processing XmlSchema object will add some overhead and result into
>>>>> >> performance. Axis2 already has JSON support main objective of this
>>>>> >> project is to provide a high performance JSON implementation,
>>>>> >> processing XmlSchema object with every response does not help to
>>>>> meet
>>>>> >> your expectations. Instead I suggest to process XmlSchema object
>>>>> only
>>>>> >> one-time during the deployment after the AxisService creation and
>>>>> >> store result of XmlSchema processing with AxisService object as a
>>>>> >> light-weight object ( Let's say a map) and use this light weight
>>>>> map
>>>>> >> with response. In this approach overhead if relatively low.
>>>>> >
>>>>> >
>>>>> > This is a POC level approach to convert xml stream to json using xml
>>>>> schema.
>>>>> > For an example there are some techniques like
>>>>> > bagger fish and others to address this problem. But those things
>>>>> complicated
>>>>> > the produced json stream.
>>>>> >
>>>>> > Once it proves at the POC level we can think of improving
>>>>> performance.
>>>>> >
>>>>> > If you look at the current axis2 json support does that properly
>>>>> address the
>>>>> > problem shameera mentioned earlier? i.e serialising the
>>>>> > array elements when there is on element. And also can that be used
>>>>> with some
>>>>> > thing like ADB or WSO2 Data Services,rules etc..
>>>>> > Idea here to develop a technique to address those problems properly
>>>>> while
>>>>> > converting an xml stream to json stream.
>>>>>
>>>>> I don't have any issue with the technique proposed here (i.e.
>>>>> processing XMLSchema to identify structure ). My only concern is why
>>>>> we should process XMLSchema with every response because in each and
>>>>> every response we end up getting same result, why shouldn't we process
>>>>> XMLSchema only one time and reuse results ? May be as last step of
>>>>> deployment or with very first response processing ?
>>>>>
>>>>> Of course we can keep it as future improvement for the moment but
>>>>> Axis2 having good history of forgetting those improvement after
>>>>> initial development every time when I dig into code I found couple of
>>>>> such TODOs.
>>>>>
>>>>
>>>> :) I am not telling to do it after GSoc project. But first we need to
>>>> have a proper POC.
>>>>
>>>> thanks,
>>>> Amila.
>>>>
>>>>>
>>>>> Thanks !
>>>>>
>>>>>
>>>>>
>>>>> >
>>>>> > thanks,
>>>>> > Amila.
>>>>> >
>>>>> >>
>>>>> >>
>>>>> >> Thanks !
>>>>> >>
>>>>> >> > use a queue to keep the schema structure. At the moment i have
>>>>> only
>>>>> >> > implemented
>>>>> >> > writeStartElement(String localName) , writeCharaters(String text)
>>>>> ,
>>>>> >> > writeEndElement()
>>>>> >> > and writeEndDocument() functions. I am processing the queue which
>>>>> has
>>>>> >> > the schema structure and put it to another stack to identify the
>>>>> scope
>>>>> >> > of
>>>>> >> > end elements.
>>>>> >> >
>>>>> >> > I am now working on, improving this to support xml which has
>>>>> complex
>>>>> >> > type
>>>>> >> > elements
>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean while.
>>>>> >> >
>>>>> >> > Thanks,
>>>>> >> > Shameera.
>>>>> >> >
>>>>> >> > --
>>>>> >> > Shameera Rathnayaka
>>>>> >> > Undergraduate
>>>>> >> > Department of Computer Science and Engineering
>>>>> >> > University of Moratuwa.
>>>>> >> > Sri Lanka.
>>>>> >> >
>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Sagara Gunathunga
>>>>> >>
>>>>> >> Blog - http://ssagara.blogspot.com
>>>>> >> Web - http://people.apache.org/~sagara/
>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>> >>
>>>>> >>
>>>>> ---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>> >>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Amila Suriarachchi
>>>>> > WSO2 Inc.
>>>>> > blog: http://amilachinthaka.blogspot.com/
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sagara Gunathunga
>>>>>
>>>>> Blog - http://ssagara.blogspot.com
>>>>> Web - http://people.apache.org/~sagara/
>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Amila Suriarachchi
>>>> WSO2 Inc.
>>>> blog: http://amilachinthaka.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Shameera Rathnayaka
>>> Undergraduate
>>> Department of Computer Science and Engineering
>>> University of Moratuwa.
>>> Sri Lanka.
>>>
>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Amila Suriarachchi
>> WSO2 Inc.
>> blog: http://amilachinthaka.blogspot.com/
>>
>
>
>
> --
> Shameera Rathnayaka
> Undergraduate
> Department of Computer Science and Engineering
> University of Moratuwa.
> Sri Lanka.
>
> Blog : http://shameerarathnayaka.blogspot.com/
>
>
--
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.
Blog : http://shameerarathnayaka.blogspot.com/
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi Sagara,
Thanks for your feedback, Yes i have implemented 2nd approach as same as
you have suggested. It only process XMLSchema at the first message. Here i
have used intermediate representation to keep the xmlSchema structure
therefore i do not need to process XMLSchema for every request but only
very first request. I store this intermediate representation in
configuration context with method Qname as a property ,yes sound is not
good there may be a memory problem with large XMLSchema, and will use it to
process next requests.
Thanks,
Shameera.
On Sat, Aug 11, 2012 at 7:38 PM, Sagara Gunathunga <
sagara.gunathunga@gmail.com> wrote:
> It seems your 1st approach provides much better performance than 2nd
> approach, can you incorporate improvements that I mentioned here[1] ?
> I believe applying those modification will solve performance issue
> with 2nd approach.
>
> [1] - http://markmail.org/message/xyfxzn2clc3r5zfg
>
> Thanks !
>
> On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
> <sh...@gmail.com> wrote:
> > Hi devs,
> >
> > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON
> > first approach(pure json) and 4. JSON second
> > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
> > summarize the results i got.
> > I used same POJO service and send the exact same request in relevent
> > convention.
> >
> > Environment:
> > OS : Ubuntu 11.04
> > JDK 1.6_30
> > memory 4 GB
> >
> > SOAP JSON badgerfish
> > JSON 1st approach JSON 2nd approach
> >
> > Concurrency Level: 50 50
> > 50 50
> > Time taken for tests: 243.13 sec 287.18 sec
> > 118.99 200.0
> > Complete requests: 500000 500000
> > 500000 500000
> > Total transferred: 2.2 GB 34.42 GB
> > 0.61 GB 0.63 GB
> > Requests per second: 2056.53 #/sec 1741.04 #/sec
> 4201.69
> > #/sec 2499.91 #/sec
> > Time per request: 24.3 ms 28.7 ms
> > 11.90 ms 20.00 ms
> > Time per request:(mean) 0.486 ms 0.544 ms
> 0.238
> > ms 0.4 ms
> >
> > Thanks,
> > Shameera.
> >
> >
> > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
> > <sh...@gmail.com> wrote:
> >>
> >> HI devs,
> >>
> >> I finished the implementation and did a performance test with existing
> >> SOAP and implemented JSON. Here is the summary of the test results.
> >> For this i used same service and send the same request in both SOAP and
> >> JSON ways, size of JSON message is lower than SOAP message but requests
> are
> >> exactly same.
> >>
> >> Environment :
> >> OS - Ubuntu 11.10
> >> Axis2 1.7(current trunk)
> >> Tool - Java benchmark
> >>
> >> num of requests : 50000
> >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
> >> Time taken for test : with SOAP 178 sec and with JSON 140 sec
> >> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000
> >> bytes
> >> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON
> >> 3,568.89 [#/sec] (mean)
> >>
> >> will do a performance test with concurrency level 50 and num of requests
> >> 500000 and give the summary of results.
> >>
> >> Thanks,
> >> Shameera.
> >>
> >>
> >>
> >>
> >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
> >> <sh...@gmail.com> wrote:
> >>>
> >>> HI devs,
> >>>
> >>> I have implemented XMLStreamWriter as well as XMLStreamReader to
> convert
> >>> XML <----> JSON with XMLSchema,
> >>> Basically it works fine, but still need few test to be done after that
> I
> >>> am willing to write performance test with implemented JSON support with
> >>> existing SOAP support.
> >>>
> >>> Thanks,
> >>> Shameera.
> >>>
> >>>
> >>>
> >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
> >>> <am...@gmail.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
> >>>> <sh...@gmail.com> wrote:
> >>>>>
> >>>>> Hi devs,
> >>>>>
> >>>>> I implemented this feature completely to convert XML to JSON with the
> >>>>> help of XmlSchema of that XML document. I tested this with different
> >>>>> Schemas, it is working fine. I implemented this as a separate
> project, so i
> >>>>> will integrate this with axis2 trunk and try with actual scenario.
> >>>>
> >>>>
> >>>> Can you use this technique at the reader level as well to support
> >>>> namespaces?
> >>>>
> >>>> Then do a performance comparison using a POJO service with the exiting
> >>>> json support and based the new gson based XMLstream API. According to
> the
> >>>> result we need to optimise the later to have better performance.
> >>>>
> >>>> thanks,
> >>>> Amila.
> >>>>
> >>>>>
> >>>>>
> >>>>> Yes, This may be a standard way to convert XML to JSON conversion as
> >>>>> other implementations don't convert JSON message correctly when it
> has one
> >>>>> value array, unless implementation supports special XML structure
> where
> >>>>> Json-lib does.
> >>>>>
> >>>>> Definitely we can improve the performance of this implementation. May
> >>>>> be in the way that Sagara had suggested in his reply or in another
> way.
> >>>>>
> >>>>> Thanks,
> >>>>> Shameera.
> >>>>>
> >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
> >>>>> <am...@gmail.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
> >>>>>> <sa...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
> >>>>>>> <am...@gmail.com> wrote:
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
> >>>>>>> > <sa...@gmail.com> wrote:
> >>>>>>> >>
> >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
> >>>>>>> >> <sh...@gmail.com> wrote:
> >>>>>>> >> > Hi devs,
> >>>>>>> >> >
> >>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I
> >>>>>>> >> > have
> >>>>>>> >> > implemented
> >>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and
> gson
> >>>>>>> >> > together
> >>>>>>> >> > to convert XML ---> JSON. I have made my progress up to
> support
> >>>>>>> >> > xml
> >>>>>>> >> > which
> >>>>>>> >> > don't have complex type elements with maxOccur >1, but may
> have
> >>>>>>> >> > simple
> >>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point
> >>>>>>> >> > of view,
> >>>>>>> >> > my
> >>>>>>> >> > present implementation supports XML --> JSON where there is
> not
> >>>>>>> >> > any JSON
> >>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
> >>>>>>> >> >
> >>>>>>> >> > Here is a summary of the implementation.
> >>>>>>> >> >
> >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
> >>>>>>> >> > constructor
> >>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
> >>>>>>> >> > XmlSchema and
> >>>>>>> >>
> >>>>>>> >> Are you plan to process XmlSchema object with every response ?
> If
> >>>>>>> >> so
> >>>>>>> >> processing XmlSchema object will add some overhead and result
> >>>>>>> >> into
> >>>>>>> >> performance. Axis2 already has JSON support main objective of
> >>>>>>> >> this
> >>>>>>> >> project is to provide a high performance JSON implementation,
> >>>>>>> >> processing XmlSchema object with every response does not help
> to
> >>>>>>> >> meet
> >>>>>>> >> your expectations. Instead I suggest to process XmlSchema
> object
> >>>>>>> >> only
> >>>>>>> >> one-time during the deployment after the AxisService creation
> and
> >>>>>>> >> store result of XmlSchema processing with AxisService object
> as
> >>>>>>> >> a
> >>>>>>> >> light-weight object ( Let's say a map) and use this light
> weight
> >>>>>>> >> map
> >>>>>>> >> with response. In this approach overhead if relatively low.
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > This is a POC level approach to convert xml stream to json using
> >>>>>>> > xml schema.
> >>>>>>> > For an example there are some techniques like
> >>>>>>> > bagger fish and others to address this problem. But those things
> >>>>>>> > complicated
> >>>>>>> > the produced json stream.
> >>>>>>> >
> >>>>>>> > Once it proves at the POC level we can think of improving
> >>>>>>> > performance.
> >>>>>>> >
> >>>>>>> > If you look at the current axis2 json support does that properly
> >>>>>>> > address the
> >>>>>>> > problem shameera mentioned earlier? i.e serialising the
> >>>>>>> > array elements when there is on element. And also can that be
> used
> >>>>>>> > with some
> >>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
> >>>>>>> > Idea here to develop a technique to address those problems
> properly
> >>>>>>> > while
> >>>>>>> > converting an xml stream to json stream.
> >>>>>>>
> >>>>>>> I don't have any issue with the technique proposed here (i.e.
> >>>>>>> processing XMLSchema to identify structure ). My only concern is
> why
> >>>>>>> we should process XMLSchema with every response because in each and
> >>>>>>> every response we end up getting same result, why shouldn't we
> >>>>>>> process
> >>>>>>> XMLSchema only one time and reuse results ? May be as last step of
> >>>>>>> deployment or with very first response processing ?
> >>>>>>>
> >>>>>>> Of course we can keep it as future improvement for the moment but
> >>>>>>> Axis2 having good history of forgetting those improvement after
> >>>>>>> initial development every time when I dig into code I found couple
> of
> >>>>>>> such TODOs.
> >>>>>>
> >>>>>>
> >>>>>> :) I am not telling to do it after GSoc project. But first we need
> to
> >>>>>> have a proper POC.
> >>>>>>
> >>>>>> thanks,
> >>>>>> Amila.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks !
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> >
> >>>>>>> > thanks,
> >>>>>>> > Amila.
> >>>>>>> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> Thanks !
> >>>>>>> >>
> >>>>>>> >> > use a queue to keep the schema structure. At the moment i have
> >>>>>>> >> > only
> >>>>>>> >> > implemented
> >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
> >>>>>>> >> > text) ,
> >>>>>>> >> > writeEndElement()
> >>>>>>> >> > and writeEndDocument() functions. I am processing the queue
> >>>>>>> >> > which has
> >>>>>>> >> > the schema structure and put it to another stack to identify
> the
> >>>>>>> >> > scope
> >>>>>>> >> > of
> >>>>>>> >> > end elements.
> >>>>>>> >> >
> >>>>>>> >> > I am now working on, improving this to support xml which has
> >>>>>>> >> > complex
> >>>>>>> >> > type
> >>>>>>> >> > elements
> >>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean
> while.
> >>>>>>> >> >
> >>>>>>> >> > Thanks,
> >>>>>>> >> > Shameera.
> >>>>>>> >> >
> >>>>>>> >> > --
> >>>>>>> >> > Shameera Rathnayaka
> >>>>>>> >> > Undergraduate
> >>>>>>> >> > Department of Computer Science and Engineering
> >>>>>>> >> > University of Moratuwa.
> >>>>>>> >> > Sri Lanka.
> >>>>>>> >> >
> >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>> >> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> --
> >>>>>>> >> Sagara Gunathunga
> >>>>>>> >>
> >>>>>>> >> Blog - http://ssagara.blogspot.com
> >>>>>>> >> Web - http://people.apache.org/~sagara/
> >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >>
> ---------------------------------------------------------------------
> >>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>> >>
> >>>>>>> >
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > --
> >>>>>>> > Amila Suriarachchi
> >>>>>>> > WSO2 Inc.
> >>>>>>> > blog: http://amilachinthaka.blogspot.com/
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Sagara Gunathunga
> >>>>>>>
> >>>>>>> Blog - http://ssagara.blogspot.com
> >>>>>>> Web - http://people.apache.org/~sagara/
> >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Amila Suriarachchi
> >>>>>> WSO2 Inc.
> >>>>>> blog: http://amilachinthaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Shameera Rathnayaka
> >>>>> Undergraduate
> >>>>> Department of Computer Science and Engineering
> >>>>> University of Moratuwa.
> >>>>> Sri Lanka.
> >>>>>
> >>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Amila Suriarachchi
> >>>> WSO2 Inc.
> >>>> blog: http://amilachinthaka.blogspot.com/
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Shameera Rathnayaka
> >>> Undergraduate
> >>> Department of Computer Science and Engineering
> >>> University of Moratuwa.
> >>> Sri Lanka.
> >>>
> >>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>
> >>
> >>
> >> --
> >> Shameera Rathnayaka
> >> Undergraduate
> >> Department of Computer Science and Engineering
> >> University of Moratuwa.
> >> Sri Lanka.
> >>
> >> Blog : http://shameerarathnayaka.blogspot.com/
> >>
> >
> >
> >
> > --
> > Shameera Rathnayaka
> > Undergraduate
> > Department of Computer Science and Engineering
> > University of Moratuwa.
> > Sri Lanka.
> >
> > Blog : http://shameerarathnayaka.blogspot.com/
> >
>
>
>
> --
> Sagara Gunathunga
>
> Blog - http://ssagara.blogspot.com
> Web - http://people.apache.org/~sagara/
> LinkedIn - http://www.linkedin.com/in/ssagara
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>
--
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.
Blog : http://shameerarathnayaka.blogspot.com/
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
Posted by Amila Suriarachchi <am...@gmail.com>.
On Sat, Aug 11, 2012 at 9:08 PM, Sagara Gunathunga <
sagara.gunathunga@gmail.com> wrote:
> It seems your 1st approach provides much better performance than 2nd
> approach, can you incorporate improvements that I mentioned here[1] ?
> I believe applying those modification will solve performance issue
> with 2nd approach.
>
When comparing the new JSON approaches which has much better performance
than the existing JSON approach.
If we compare new two approaches pure json approach is only for POJO
services. But the second approach can be used with any service written top
of xml stream API.
So we need to merge these two so that users can configure to use what they
need.
thanks,
Amila.
>
> [1] - http://markmail.org/message/xyfxzn2clc3r5zfg
>
> Thanks !
>
> On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
> <sh...@gmail.com> wrote:
> > Hi devs,
> >
> > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON
> > first approach(pure json) and 4. JSON second
> > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
> > summarize the results i got.
> > I used same POJO service and send the exact same request in relevent
> > convention.
> >
> > Environment:
> > OS : Ubuntu 11.04
> > JDK 1.6_30
> > memory 4 GB
> >
> > SOAP JSON badgerfish
> > JSON 1st approach JSON 2nd approach
> >
> > Concurrency Level: 50 50
> > 50 50
> > Time taken for tests: 243.13 sec 287.18 sec
> > 118.99 200.0
> > Complete requests: 500000 500000
> > 500000 500000
> > Total transferred: 2.2 GB 34.42 GB
> > 0.61 GB 0.63 GB
> > Requests per second: 2056.53 #/sec 1741.04 #/sec
> 4201.69
> > #/sec 2499.91 #/sec
> > Time per request: 24.3 ms 28.7 ms
> > 11.90 ms 20.00 ms
> > Time per request:(mean) 0.486 ms 0.544 ms
> 0.238
> > ms 0.4 ms
> >
> > Thanks,
> > Shameera.
> >
> >
> > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
> > <sh...@gmail.com> wrote:
> >>
> >> HI devs,
> >>
> >> I finished the implementation and did a performance test with existing
> >> SOAP and implemented JSON. Here is the summary of the test results.
> >> For this i used same service and send the same request in both SOAP and
> >> JSON ways, size of JSON message is lower than SOAP message but requests
> are
> >> exactly same.
> >>
> >> Environment :
> >> OS - Ubuntu 11.10
> >> Axis2 1.7(current trunk)
> >> Tool - Java benchmark
> >>
> >> num of requests : 50000
> >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
> >> Time taken for test : with SOAP 178 sec and with JSON 140 sec
> >> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000
> >> bytes
> >> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON
> >> 3,568.89 [#/sec] (mean)
> >>
> >> will do a performance test with concurrency level 50 and num of requests
> >> 500000 and give the summary of results.
> >>
> >> Thanks,
> >> Shameera.
> >>
> >>
> >>
> >>
> >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
> >> <sh...@gmail.com> wrote:
> >>>
> >>> HI devs,
> >>>
> >>> I have implemented XMLStreamWriter as well as XMLStreamReader to
> convert
> >>> XML <----> JSON with XMLSchema,
> >>> Basically it works fine, but still need few test to be done after that
> I
> >>> am willing to write performance test with implemented JSON support with
> >>> existing SOAP support.
> >>>
> >>> Thanks,
> >>> Shameera.
> >>>
> >>>
> >>>
> >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
> >>> <am...@gmail.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
> >>>> <sh...@gmail.com> wrote:
> >>>>>
> >>>>> Hi devs,
> >>>>>
> >>>>> I implemented this feature completely to convert XML to JSON with the
> >>>>> help of XmlSchema of that XML document. I tested this with different
> >>>>> Schemas, it is working fine. I implemented this as a separate
> project, so i
> >>>>> will integrate this with axis2 trunk and try with actual scenario.
> >>>>
> >>>>
> >>>> Can you use this technique at the reader level as well to support
> >>>> namespaces?
> >>>>
> >>>> Then do a performance comparison using a POJO service with the exiting
> >>>> json support and based the new gson based XMLstream API. According to
> the
> >>>> result we need to optimise the later to have better performance.
> >>>>
> >>>> thanks,
> >>>> Amila.
> >>>>
> >>>>>
> >>>>>
> >>>>> Yes, This may be a standard way to convert XML to JSON conversion as
> >>>>> other implementations don't convert JSON message correctly when it
> has one
> >>>>> value array, unless implementation supports special XML structure
> where
> >>>>> Json-lib does.
> >>>>>
> >>>>> Definitely we can improve the performance of this implementation. May
> >>>>> be in the way that Sagara had suggested in his reply or in another
> way.
> >>>>>
> >>>>> Thanks,
> >>>>> Shameera.
> >>>>>
> >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
> >>>>> <am...@gmail.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
> >>>>>> <sa...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
> >>>>>>> <am...@gmail.com> wrote:
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
> >>>>>>> > <sa...@gmail.com> wrote:
> >>>>>>> >>
> >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
> >>>>>>> >> <sh...@gmail.com> wrote:
> >>>>>>> >> > Hi devs,
> >>>>>>> >> >
> >>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I
> >>>>>>> >> > have
> >>>>>>> >> > implemented
> >>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and
> gson
> >>>>>>> >> > together
> >>>>>>> >> > to convert XML ---> JSON. I have made my progress up to
> support
> >>>>>>> >> > xml
> >>>>>>> >> > which
> >>>>>>> >> > don't have complex type elements with maxOccur >1, but may
> have
> >>>>>>> >> > simple
> >>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point
> >>>>>>> >> > of view,
> >>>>>>> >> > my
> >>>>>>> >> > present implementation supports XML --> JSON where there is
> not
> >>>>>>> >> > any JSON
> >>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
> >>>>>>> >> >
> >>>>>>> >> > Here is a summary of the implementation.
> >>>>>>> >> >
> >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
> >>>>>>> >> > constructor
> >>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
> >>>>>>> >> > XmlSchema and
> >>>>>>> >>
> >>>>>>> >> Are you plan to process XmlSchema object with every response ?
> If
> >>>>>>> >> so
> >>>>>>> >> processing XmlSchema object will add some overhead and result
> >>>>>>> >> into
> >>>>>>> >> performance. Axis2 already has JSON support main objective of
> >>>>>>> >> this
> >>>>>>> >> project is to provide a high performance JSON implementation,
> >>>>>>> >> processing XmlSchema object with every response does not help
> to
> >>>>>>> >> meet
> >>>>>>> >> your expectations. Instead I suggest to process XmlSchema
> object
> >>>>>>> >> only
> >>>>>>> >> one-time during the deployment after the AxisService creation
> and
> >>>>>>> >> store result of XmlSchema processing with AxisService object
> as
> >>>>>>> >> a
> >>>>>>> >> light-weight object ( Let's say a map) and use this light
> weight
> >>>>>>> >> map
> >>>>>>> >> with response. In this approach overhead if relatively low.
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > This is a POC level approach to convert xml stream to json using
> >>>>>>> > xml schema.
> >>>>>>> > For an example there are some techniques like
> >>>>>>> > bagger fish and others to address this problem. But those things
> >>>>>>> > complicated
> >>>>>>> > the produced json stream.
> >>>>>>> >
> >>>>>>> > Once it proves at the POC level we can think of improving
> >>>>>>> > performance.
> >>>>>>> >
> >>>>>>> > If you look at the current axis2 json support does that properly
> >>>>>>> > address the
> >>>>>>> > problem shameera mentioned earlier? i.e serialising the
> >>>>>>> > array elements when there is on element. And also can that be
> used
> >>>>>>> > with some
> >>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
> >>>>>>> > Idea here to develop a technique to address those problems
> properly
> >>>>>>> > while
> >>>>>>> > converting an xml stream to json stream.
> >>>>>>>
> >>>>>>> I don't have any issue with the technique proposed here (i.e.
> >>>>>>> processing XMLSchema to identify structure ). My only concern is
> why
> >>>>>>> we should process XMLSchema with every response because in each and
> >>>>>>> every response we end up getting same result, why shouldn't we
> >>>>>>> process
> >>>>>>> XMLSchema only one time and reuse results ? May be as last step of
> >>>>>>> deployment or with very first response processing ?
> >>>>>>>
> >>>>>>> Of course we can keep it as future improvement for the moment but
> >>>>>>> Axis2 having good history of forgetting those improvement after
> >>>>>>> initial development every time when I dig into code I found couple
> of
> >>>>>>> such TODOs.
> >>>>>>
> >>>>>>
> >>>>>> :) I am not telling to do it after GSoc project. But first we need
> to
> >>>>>> have a proper POC.
> >>>>>>
> >>>>>> thanks,
> >>>>>> Amila.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks !
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> >
> >>>>>>> > thanks,
> >>>>>>> > Amila.
> >>>>>>> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> Thanks !
> >>>>>>> >>
> >>>>>>> >> > use a queue to keep the schema structure. At the moment i have
> >>>>>>> >> > only
> >>>>>>> >> > implemented
> >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
> >>>>>>> >> > text) ,
> >>>>>>> >> > writeEndElement()
> >>>>>>> >> > and writeEndDocument() functions. I am processing the queue
> >>>>>>> >> > which has
> >>>>>>> >> > the schema structure and put it to another stack to identify
> the
> >>>>>>> >> > scope
> >>>>>>> >> > of
> >>>>>>> >> > end elements.
> >>>>>>> >> >
> >>>>>>> >> > I am now working on, improving this to support xml which has
> >>>>>>> >> > complex
> >>>>>>> >> > type
> >>>>>>> >> > elements
> >>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean
> while.
> >>>>>>> >> >
> >>>>>>> >> > Thanks,
> >>>>>>> >> > Shameera.
> >>>>>>> >> >
> >>>>>>> >> > --
> >>>>>>> >> > Shameera Rathnayaka
> >>>>>>> >> > Undergraduate
> >>>>>>> >> > Department of Computer Science and Engineering
> >>>>>>> >> > University of Moratuwa.
> >>>>>>> >> > Sri Lanka.
> >>>>>>> >> >
> >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>> >> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> --
> >>>>>>> >> Sagara Gunathunga
> >>>>>>> >>
> >>>>>>> >> Blog - http://ssagara.blogspot.com
> >>>>>>> >> Web - http://people.apache.org/~sagara/
> >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >>
> ---------------------------------------------------------------------
> >>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>> >>
> >>>>>>> >
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > --
> >>>>>>> > Amila Suriarachchi
> >>>>>>> > WSO2 Inc.
> >>>>>>> > blog: http://amilachinthaka.blogspot.com/
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Sagara Gunathunga
> >>>>>>>
> >>>>>>> Blog - http://ssagara.blogspot.com
> >>>>>>> Web - http://people.apache.org/~sagara/
> >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>>
> >>>>>>>
> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Amila Suriarachchi
> >>>>>> WSO2 Inc.
> >>>>>> blog: http://amilachinthaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Shameera Rathnayaka
> >>>>> Undergraduate
> >>>>> Department of Computer Science and Engineering
> >>>>> University of Moratuwa.
> >>>>> Sri Lanka.
> >>>>>
> >>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Amila Suriarachchi
> >>>> WSO2 Inc.
> >>>> blog: http://amilachinthaka.blogspot.com/
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Shameera Rathnayaka
> >>> Undergraduate
> >>> Department of Computer Science and Engineering
> >>> University of Moratuwa.
> >>> Sri Lanka.
> >>>
> >>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>
> >>
> >>
> >> --
> >> Shameera Rathnayaka
> >> Undergraduate
> >> Department of Computer Science and Engineering
> >> University of Moratuwa.
> >> Sri Lanka.
> >>
> >> Blog : http://shameerarathnayaka.blogspot.com/
> >>
> >
> >
> >
> > --
> > Shameera Rathnayaka
> > Undergraduate
> > Department of Computer Science and Engineering
> > University of Moratuwa.
> > Sri Lanka.
> >
> > Blog : http://shameerarathnayaka.blogspot.com/
> >
>
>
>
> --
> Sagara Gunathunga
>
> Blog - http://ssagara.blogspot.com
> Web - http://people.apache.org/~sagara/
> LinkedIn - http://www.linkedin.com/in/ssagara
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>
--
Amila Suriarachchi
WSO2 Inc.
blog: http://amilachinthaka.blogspot.com/
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi devs ,
According to the Axis2 architecture, It only uses content type and
inputStream to build the SOAP message in the message builder. But with my
second approach i have used another major part of the Axis2 architecture,
which is handler, to build the complete message using XmlSchema of the
relevant operataion. With the xmlschema of the operation, i can get the
namespace uri for each element. Therefore I have improved my implementation
to return correct namespace uri according to the xmlSchema too. With this
implementation i would say that my second approach support namespaces with
pure JSON requests.
After improving the code i did a performance test with ADB generated
skeleton. Here is the summary of the results. updated my patch with this
improvement.
Here is the summary of results,
*description ADB request
pure JSON requset with 2nd Approach
*Document Length: 1780
bytes 678 bytes
Concurrency Level:
50 50
Time taken for tests: 159.9 sec
138.4 sec
Complete requests:
500000 500000
Requests per second: 3,126.79 [#/sec]
(mean) 3,611.68 [#/sec] (mean)
Time per request: 0.320
[ms] 0.277 [ms] (mean, across all
conc req)
Thanks,
Shameera.
On Sun, Aug 12, 2012 at 5:00 PM, Shameera Rathnayaka <shameerainfo@gmail.com
> wrote:
> Hi Martin,
>
> On Sun, Aug 12, 2012 at 7:36 AM, Martin Gainty <mg...@hotmail.com>wrote:
>
>> It seems we lose alot of meta data by casting off the SOAP 1.1
>> superstructure a partial list of what we lose would include:
>> ability to configure to differing styles (JIBX-RI vs ADB vs XMLBeans)
>> ability to adhere to different protocols (SMTP and or FTP and or HTTP 1.1)
>> Ability to incorporate differing XSD's
>> Parser for both the Input and Output Message Receivers (MEP) goes away
>>
>> I would like to know what advantages people see with using strictly JSON
>> over a SOAP formatted sender and receiver
>>
>
> If you use JavaScript to call service which is a POJO service and hosted
> in Axis2. Then you need convert you JSON object to Badgerfish format or use
> mapped format, even though it is more slower than normal SOAP way as it
> implicitly convert to XML String before process it. With above first
> approach you can send pure JSON string to the same Axis2 service. It is not
> convert to XML in order to process.
>
> In this second approach introduce XMLStreamReader/XMLSreamWriter
> implementations for JSON, it is not convert to XML implicitly , but provide
> relevant XML inforset processing JSON inputStream. Therefore it can be used
> with any service written top of xml stream API not only POJO like first
> approach.
>
> As you can see both approaches have better performance than existing SOAP
> and JSON supports.
>
>
> Thanks,
> Shameera.
>
>>
>> thanks!
>> Martin
>> ______________________________________________
>> Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
>>
>> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene
>> Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte
>> Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht
>> dient lediglich dem Austausch von Informationen und entfaltet keine
>> rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von
>> E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
>>
>> Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni.
>>
>>
>>
>>
>> > From: sagara.gunathunga@gmail.com
>> > Date: Sat, 11 Aug 2012 19:38:16 +0530
>>
>> > Subject: Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter
>> API and Schemas with google-gson
>> > To: java-dev@axis.apache.org
>> >
>> > It seems your 1st approach provides much better performance than 2nd
>> > approach, can you incorporate improvements that I mentioned here[1] ?
>> > I believe applying those modification will solve performance issue
>> > with 2nd approach.
>> >
>> > [1] - http://markmail.org/message/xyfxzn2clc3r5zfg
>> >
>> > Thanks !
>> >
>> > On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
>> > <sh...@gmail.com> wrote:
>> > > Hi devs,
>> > >
>> > > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3
>> JSON
>> > > first approach(pure json) and 4. JSON second
>> > > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
>> > > summarize the results i got.
>> > > I used same POJO service and send the exact same request in relevent
>> > > convention.
>> > >
>> > > Environment:
>> > > OS : Ubuntu 11.04
>> > > JDK 1.6_30
>> > > memory 4 GB
>> > >
>> > > SOAP JSON badgerfish
>> > > JSON 1st approach JSON 2nd approach
>> > >
>> > > Concurrency Level: 50 50
>> > > 50 50
>> > > Time taken for tests: 243.13 sec 287.18 sec
>> > > 118.99 200.0
>> > > Complete requests: 500000 500000
>> > > 500000 500000
>> > > Total transferred: 2.2 GB 34.42 GB
>> > > 0.61 GB 0.63 GB
>> > > Requests per second: 2056.53 #/sec 1741.04 #/sec 4201.69
>> > > #/sec 2499.91 #/sec
>> > > Time per request: 24.3 ms 28.7 ms
>> > > 11.90 ms 20.00 ms
>> > > Time per request:(mean) 0.486 ms 0.544 ms 0.238
>> > > ms 0.4 ms
>> > >
>> > > Thanks,
>> > > Shameera.
>> > >
>> > >
>> > > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
>> > > <sh...@gmail.com> wrote:
>> > >>
>> > >> HI devs,
>> > >>
>> > >> I finished the implementation and did a performance test with
>> existing
>> > >> SOAP and implemented JSON. Here is the summary of the test results.
>> > >> For this i used same service and send the same request in both SOAP
>> and
>> > >> JSON ways, size of JSON message is lower than SOAP message but
>> requests are
>> > >> exactly same.
>> > >>
>> > >> Environment :
>> > >> OS - Ubuntu 11.10
>> > >> Axis2 1.7(current trunk)
>> > >> Tool - Java benchmark
>> > >>
>> > >> num of requests : 50000
>> > >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
>> > >> Time taken for test : with SOAP 178 sec and with JSON 140 sec
>> > >> Total transferred : with SOAP 2371000000 bytes and with JSON
>> 674000000
>> > >> bytes
>> > >> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON
>> > >> 3,568.89 [#/sec] (mean)
>> > >>
>> > >> will do a performance test with concurrency level 50 and num of
>> requests
>> > >> 500000 and give the summary of results.
>> > >>
>> > >> Thanks,
>> > >> Shameera.
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
>> > >> <sh...@gmail.com> wrote:
>> > >>>
>> > >>> HI devs,
>> > >>>
>> > >>> I have implemented XMLStreamWriter as well as XMLStreamReader to
>> convert
>> > >>> XML <----> JSON with XMLSchema,
>> > >>> Basically it works fine, but still need few test to be done after
>> that I
>> > >>> am willing to write performance test with implemented JSON support
>> with
>> > >>> existing SOAP support.
>> > >>>
>> > >>> Thanks,
>> > >>> Shameera.
>> > >>>
>> > >>>
>> > >>>
>> > >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
>> > >>> <am...@gmail.com> wrote:
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
>> > >>>> <sh...@gmail.com> wrote:
>> > >>>>>
>> > >>>>> Hi devs,
>> > >>>>>
>> > >>>>> I implemented this feature completely to convert XML to JSON with
>> the
>> > >>>>> help of XmlSchema of that XML document. I tested this with
>> different
>> > >>>>> Schemas, it is working fine. I implemented this as a separate
>> project, so i
>> > >>>>> will integrate this with axis2 trunk and try with actual scenario.
>> > >>>>
>> > >>>>
>> > >>>> Can you use this technique at the reader level as well to support
>> > >>>> namespaces?
>> > >>>>
>> > >>>> Then do a performance comparison using a POJO service with the
>> exiting
>> > >>>> json support and based the new gson based XMLstream API. According
>> to the
>> > >>>> result we need to optimise the later to have better performance.
>> > >>>>
>> > >>>> thanks,
>> > >>>> Amila.
>> > >>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> Yes, This may be a standard way to convert XML to JSON conversion
>> as
>> > >>>>> other implementations don't convert JSON message correctly when
>> it has one
>> > >>>>> value array, unless implementation supports special XML structure
>> where
>> > >>>>> Json-lib does.
>> > >>>>>
>> > >>>>> Definitely we can improve the performance of this implementation.
>> May
>> > >>>>> be in the way that Sagara had suggested in his reply or in
>> another way.
>> > >>>>>
>> > >>>>> Thanks,
>> > >>>>> Shameera.
>> > >>>>>
>> > >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
>> > >>>>> <am...@gmail.com> wrote:
>> > >>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
>> > >>>>>> <sa...@gmail.com> wrote:
>> > >>>>>>>
>> > >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
>> > >>>>>>> <am...@gmail.com> wrote:
>> > >>>>>>> >
>> > >>>>>>> >
>> > >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
>> > >>>>>>> > <sa...@gmail.com> wrote:
>> > >>>>>>> >>
>> > >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
>> > >>>>>>> >> <sh...@gmail.com> wrote:
>> > >>>>>>> >> > Hi devs,
>> > >>>>>>> >> >
>> > >>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] ,
>> I
>> > >>>>>>> >> > have
>> > >>>>>>> >> > implemented
>> > >>>>>>> >> > XMLStreamWriter API to use schema of the processing xml
>> and gson
>> > >>>>>>> >> > together
>> > >>>>>>> >> > to convert XML ---> JSON. I have made my progress up to
>> support
>> > >>>>>>> >> > xml
>> > >>>>>>> >> > which
>> > >>>>>>> >> > don't have complex type elements with maxOccur >1, but may
>> have
>> > >>>>>>> >> > simple
>> > >>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON
>> point
>> > >>>>>>> >> > of view,
>> > >>>>>>> >> > my
>> > >>>>>>> >> > present implementation supports XML --> JSON where there
>> is not
>> > >>>>>>> >> > any JSON
>> > >>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
>> > >>>>>>> >> >
>> > >>>>>>> >> > Here is a summary of the implementation.
>> > >>>>>>> >> >
>> > >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
>> > >>>>>>> >> > constructor
>> > >>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
>> > >>>>>>> >> > XmlSchema and
>> > >>>>>>> >>
>> > >>>>>>> >> Are you plan to process XmlSchema object with every response
>> ? If
>> > >>>>>>> >> so
>> > >>>>>>> >> processing XmlSchema object will add some overhead and result
>> > >>>>>>> >> into
>> > >>>>>>> >> performance. Axis2 already has JSON support main objective of
>> > >>>>>>> >> this
>> > >>>>>>> >> project is to provide a high performance JSON implementation,
>> > >>>>>>> >> processing XmlSchema object with every response does not
>> help to
>> > >>>>>>> >> meet
>> > >>>>>>> >> your expectations. Instead I suggest to process XmlSchema
>> object
>> > >>>>>>> >> only
>> > >>>>>>> >> one-time during the deployment after the AxisService
>> creation and
>> > >>>>>>> >> store result of XmlSchema processing with AxisService object
>> as
>> > >>>>>>> >> a
>> > >>>>>>> >> light-weight object ( Let's say a map) and use this light
>> weight
>> > >>>>>>> >> map
>> > >>>>>>> >> with response. In this approach overhead if relatively low.
>> > >>>>>>> >
>> > >>>>>>> >
>> > >>>>>>> > This is a POC level approach to convert xml stream to json
>> using
>> > >>>>>>> > xml schema.
>> > >>>>>>> > For an example there are some techniques like
>> > >>>>>>> > bagger fish and others to address this problem. But those
>> things
>> > >>>>>>> > complicated
>> > >>>>>>> > the produced json stream.
>> > >>>>>>> >
>> > >>>>>>> > Once it proves at the POC level we can think of improving
>> > >>>>>>> > performance.
>> > >>>>>>> >
>> > >>>>>>> > If you look at the current axis2 json support does that
>> properly
>> > >>>>>>> > address the
>> > >>>>>>> > problem shameera mentioned earlier? i.e serialising the
>> > >>>>>>> > array elements when there is on element. And also can that be
>> used
>> > >>>>>>> > with some
>> > >>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
>> > >>>>>>> > Idea here to develop a technique to address those problems
>> properly
>> > >>>>>>> > while
>> > >>>>>>> > converting an xml stream to json stream.
>> > >>>>>>>
>> > >>>>>>> I don't have any issue with the technique proposed here (i.e.
>> > >>>>>>> processing XMLSchema to identify structure ). My only concern
>> is why
>> > >>>>>>> we should process XMLSchema with every response because in each
>> and
>> > >>>>>>> every response we end up getting same result, why shouldn't we
>> > >>>>>>> process
>> > >>>>>>> XMLSchema only one time and reuse results ? May be as last step
>> of
>> > >>>>>>> deployment or with very first response processing ?
>> > >>>>>>>
>> > >>>>>>> Of course we can keep it as future improvement for the moment
>> but
>> > >>>>>>> Axis2 having good history of forgetting those improvement after
>> > >>>>>>> initial development every time when I dig into code I found
>> couple of
>> > >>>>>>> such TODOs.
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> :) I am not telling to do it after GSoc project. But first we
>> need to
>> > >>>>>> have a proper POC.
>> > >>>>>>
>> > >>>>>> thanks,
>> > >>>>>> Amila.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> Thanks !
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> >
>> > >>>>>>> > thanks,
>> > >>>>>>> > Amila.
>> > >>>>>>> >
>> > >>>>>>> >>
>> > >>>>>>> >>
>> > >>>>>>> >> Thanks !
>> > >>>>>>> >>
>> > >>>>>>> >> > use a queue to keep the schema structure. At the moment i
>> have
>> > >>>>>>> >> > only
>> > >>>>>>> >> > implemented
>> > >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
>> > >>>>>>> >> > text) ,
>> > >>>>>>> >> > writeEndElement()
>> > >>>>>>> >> > and writeEndDocument() functions. I am processing the queue
>> > >>>>>>> >> > which has
>> > >>>>>>> >> > the schema structure and put it to another stack to
>> identify the
>> > >>>>>>> >> > scope
>> > >>>>>>> >> > of
>> > >>>>>>> >> > end elements.
>> > >>>>>>> >> >
>> > >>>>>>> >> > I am now working on, improving this to support xml which
>> has
>> > >>>>>>> >> > complex
>> > >>>>>>> >> > type
>> > >>>>>>> >> > elements
>> > >>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean
>> while.
>> > >>>>>>> >> >
>> > >>>>>>> >> > Thanks,
>> > >>>>>>> >> > Shameera.
>> > >>>>>>> >> >
>> > >>>>>>> >> > --
>> > >>>>>>> >> > Shameera Rathnayaka
>> > >>>>>>> >> > Undergraduate
>> > >>>>>>> >> > Department of Computer Science and Engineering
>> > >>>>>>> >> > University of Moratuwa.
>> > >>>>>>> >> > Sri Lanka.
>> > >>>>>>> >> >
>> > >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
>> > >>>>>>> >> >
>> > >>>>>>> >>
>> > >>>>>>> >>
>> > >>>>>>> >>
>> > >>>>>>> >> --
>> > >>>>>>> >> Sagara Gunathunga
>> > >>>>>>> >>
>> > >>>>>>> >> Blog - http://ssagara.blogspot.com
>> > >>>>>>> >> Web - http://people.apache.org/~sagara/
>> > >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
>> > >>>>>>> >>
>> > >>>>>>> >>
>> > >>>>>>> >>
>> ---------------------------------------------------------------------
>> > >>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > >>>>>>> >> For additional commands, e-mail:
>> java-dev-help@axis.apache.org
>> > >>>>>>> >>
>> > >>>>>>> >
>> > >>>>>>> >
>> > >>>>>>> >
>> > >>>>>>> > --
>> > >>>>>>> > Amila Suriarachchi
>> > >>>>>>> > WSO2 Inc.
>> > >>>>>>> > blog: http://amilachinthaka.blogspot.com/
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> --
>> > >>>>>>> Sagara Gunathunga
>> > >>>>>>>
>> > >>>>>>> Blog - http://ssagara.blogspot.com
>> > >>>>>>> Web - http://people.apache.org/~sagara/
>> > >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
>> > >>>>>>>
>> > >>>>>>>
>> ---------------------------------------------------------------------
>> > >>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > >>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> --
>> > >>>>>> Amila Suriarachchi
>> > >>>>>> WSO2 Inc.
>> > >>>>>> blog: http://amilachinthaka.blogspot.com/
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> Shameera Rathnayaka
>> > >>>>> Undergraduate
>> > >>>>> Department of Computer Science and Engineering
>> > >>>>> University of Moratuwa.
>> > >>>>> Sri Lanka.
>> > >>>>>
>> > >>>>> Blog : http://shameerarathnayaka.blogspot.com/
>> > >>>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Amila Suriarachchi
>> > >>>> WSO2 Inc.
>> > >>>> blog: http://amilachinthaka.blogspot.com/
>> > >>>
>> > >>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Shameera Rathnayaka
>> > >>> Undergraduate
>> > >>> Department of Computer Science and Engineering
>> > >>> University of Moratuwa.
>> > >>> Sri Lanka.
>> > >>>
>> > >>> Blog : http://shameerarathnayaka.blogspot.com/
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Shameera Rathnayaka
>> > >> Undergraduate
>> > >> Department of Computer Science and Engineering
>> > >> University of Moratuwa.
>> > >> Sri Lanka.
>> > >>
>> > >> Blog : http://shameerarathnayaka.blogspot.com/
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Shameera Rathnayaka
>> > > Undergraduate
>> > > Department of Computer Science and Engineering
>> > > University of Moratuwa.
>> > > Sri Lanka.
>> > >
>> > > Blog : http://shameerarathnayaka.blogspot.com/
>> > >
>> >
>> >
>> >
>> > --
>> > Sagara Gunathunga
>> >
>> > Blog - http://ssagara.blogspot.com
>> > Web - http://people.apache.org/~sagara/
>> > LinkedIn - http://www.linkedin.com/in/ssagara
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> > For additional commands, e-mail: java-dev-help@axis.apache.org
>> >
>>
>
>
>
> --
> Shameera Rathnayaka
> Undergraduate
> Department of Computer Science and Engineering
> University of Moratuwa.
> Sri Lanka.
>
> Blog : http://shameerarathnayaka.blogspot.com/
>
>
--
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.
Blog : http://shameerarathnayaka.blogspot.com/
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi Martin,
On Sun, Aug 12, 2012 at 7:36 AM, Martin Gainty <mg...@hotmail.com> wrote:
> It seems we lose alot of meta data by casting off the SOAP 1.1
> superstructure a partial list of what we lose would include:
> ability to configure to differing styles (JIBX-RI vs ADB vs XMLBeans)
> ability to adhere to different protocols (SMTP and or FTP and or HTTP 1.1)
> Ability to incorporate differing XSD's
> Parser for both the Input and Output Message Receivers (MEP) goes away
>
> I would like to know what advantages people see with using strictly JSON
> over a SOAP formatted sender and receiver
>
If you use JavaScript to call service which is a POJO service and hosted in
Axis2. Then you need convert you JSON object to Badgerfish format or use
mapped format, even though it is more slower than normal SOAP way as it
implicitly convert to XML String before process it. With above first
approach you can send pure JSON string to the same Axis2 service. It is not
convert to XML in order to process.
In this second approach introduce XMLStreamReader/XMLSreamWriter
implementations for JSON, it is not convert to XML implicitly , but provide
relevant XML inforset processing JSON inputStream. Therefore it can be used
with any service written top of xml stream API not only POJO like first
approach.
As you can see both approaches have better performance than existing SOAP
and JSON supports.
Thanks,
Shameera.
>
> thanks!
> Martin
> ______________________________________________
> Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
>
> Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene
> Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte
> Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht
> dient lediglich dem Austausch von Informationen und entfaltet keine
> rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von
> E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
>
> Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni.
>
>
>
>
> > From: sagara.gunathunga@gmail.com
> > Date: Sat, 11 Aug 2012 19:38:16 +0530
>
> > Subject: Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter
> API and Schemas with google-gson
> > To: java-dev@axis.apache.org
> >
> > It seems your 1st approach provides much better performance than 2nd
> > approach, can you incorporate improvements that I mentioned here[1] ?
> > I believe applying those modification will solve performance issue
> > with 2nd approach.
> >
> > [1] - http://markmail.org/message/xyfxzn2clc3r5zfg
> >
> > Thanks !
> >
> > On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
> > <sh...@gmail.com> wrote:
> > > Hi devs,
> > >
> > > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3
> JSON
> > > first approach(pure json) and 4. JSON second
> > > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
> > > summarize the results i got.
> > > I used same POJO service and send the exact same request in relevent
> > > convention.
> > >
> > > Environment:
> > > OS : Ubuntu 11.04
> > > JDK 1.6_30
> > > memory 4 GB
> > >
> > > SOAP JSON badgerfish
> > > JSON 1st approach JSON 2nd approach
> > >
> > > Concurrency Level: 50 50
> > > 50 50
> > > Time taken for tests: 243.13 sec 287.18 sec
> > > 118.99 200.0
> > > Complete requests: 500000 500000
> > > 500000 500000
> > > Total transferred: 2.2 GB 34.42 GB
> > > 0.61 GB 0.63 GB
> > > Requests per second: 2056.53 #/sec 1741.04 #/sec 4201.69
> > > #/sec 2499.91 #/sec
> > > Time per request: 24.3 ms 28.7 ms
> > > 11.90 ms 20.00 ms
> > > Time per request:(mean) 0.486 ms 0.544 ms 0.238
> > > ms 0.4 ms
> > >
> > > Thanks,
> > > Shameera.
> > >
> > >
> > > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
> > > <sh...@gmail.com> wrote:
> > >>
> > >> HI devs,
> > >>
> > >> I finished the implementation and did a performance test with existing
> > >> SOAP and implemented JSON. Here is the summary of the test results.
> > >> For this i used same service and send the same request in both SOAP
> and
> > >> JSON ways, size of JSON message is lower than SOAP message but
> requests are
> > >> exactly same.
> > >>
> > >> Environment :
> > >> OS - Ubuntu 11.10
> > >> Axis2 1.7(current trunk)
> > >> Tool - Java benchmark
> > >>
> > >> num of requests : 50000
> > >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
> > >> Time taken for test : with SOAP 178 sec and with JSON 140 sec
> > >> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000
> > >> bytes
> > >> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON
> > >> 3,568.89 [#/sec] (mean)
> > >>
> > >> will do a performance test with concurrency level 50 and num of
> requests
> > >> 500000 and give the summary of results.
> > >>
> > >> Thanks,
> > >> Shameera.
> > >>
> > >>
> > >>
> > >>
> > >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
> > >> <sh...@gmail.com> wrote:
> > >>>
> > >>> HI devs,
> > >>>
> > >>> I have implemented XMLStreamWriter as well as XMLStreamReader to
> convert
> > >>> XML <----> JSON with XMLSchema,
> > >>> Basically it works fine, but still need few test to be done after
> that I
> > >>> am willing to write performance test with implemented JSON support
> with
> > >>> existing SOAP support.
> > >>>
> > >>> Thanks,
> > >>> Shameera.
> > >>>
> > >>>
> > >>>
> > >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
> > >>> <am...@gmail.com> wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
> > >>>> <sh...@gmail.com> wrote:
> > >>>>>
> > >>>>> Hi devs,
> > >>>>>
> > >>>>> I implemented this feature completely to convert XML to JSON with
> the
> > >>>>> help of XmlSchema of that XML document. I tested this with
> different
> > >>>>> Schemas, it is working fine. I implemented this as a separate
> project, so i
> > >>>>> will integrate this with axis2 trunk and try with actual scenario.
> > >>>>
> > >>>>
> > >>>> Can you use this technique at the reader level as well to support
> > >>>> namespaces?
> > >>>>
> > >>>> Then do a performance comparison using a POJO service with the
> exiting
> > >>>> json support and based the new gson based XMLstream API. According
> to the
> > >>>> result we need to optimise the later to have better performance.
> > >>>>
> > >>>> thanks,
> > >>>> Amila.
> > >>>>
> > >>>>>
> > >>>>>
> > >>>>> Yes, This may be a standard way to convert XML to JSON conversion
> as
> > >>>>> other implementations don't convert JSON message correctly when it
> has one
> > >>>>> value array, unless implementation supports special XML structure
> where
> > >>>>> Json-lib does.
> > >>>>>
> > >>>>> Definitely we can improve the performance of this implementation.
> May
> > >>>>> be in the way that Sagara had suggested in his reply or in another
> way.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Shameera.
> > >>>>>
> > >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
> > >>>>> <am...@gmail.com> wrote:
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
> > >>>>>> <sa...@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
> > >>>>>>> <am...@gmail.com> wrote:
> > >>>>>>> >
> > >>>>>>> >
> > >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
> > >>>>>>> > <sa...@gmail.com> wrote:
> > >>>>>>> >>
> > >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
> > >>>>>>> >> <sh...@gmail.com> wrote:
> > >>>>>>> >> > Hi devs,
> > >>>>>>> >> >
> > >>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I
> > >>>>>>> >> > have
> > >>>>>>> >> > implemented
> > >>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and
> gson
> > >>>>>>> >> > together
> > >>>>>>> >> > to convert XML ---> JSON. I have made my progress up to
> support
> > >>>>>>> >> > xml
> > >>>>>>> >> > which
> > >>>>>>> >> > don't have complex type elements with maxOccur >1, but may
> have
> > >>>>>>> >> > simple
> > >>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON
> point
> > >>>>>>> >> > of view,
> > >>>>>>> >> > my
> > >>>>>>> >> > present implementation supports XML --> JSON where there is
> not
> > >>>>>>> >> > any JSON
> > >>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
> > >>>>>>> >> >
> > >>>>>>> >> > Here is a summary of the implementation.
> > >>>>>>> >> >
> > >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
> > >>>>>>> >> > constructor
> > >>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
> > >>>>>>> >> > XmlSchema and
> > >>>>>>> >>
> > >>>>>>> >> Are you plan to process XmlSchema object with every response
> ? If
> > >>>>>>> >> so
> > >>>>>>> >> processing XmlSchema object will add some overhead and result
> > >>>>>>> >> into
> > >>>>>>> >> performance. Axis2 already has JSON support main objective of
> > >>>>>>> >> this
> > >>>>>>> >> project is to provide a high performance JSON implementation,
> > >>>>>>> >> processing XmlSchema object with every response does not help
> to
> > >>>>>>> >> meet
> > >>>>>>> >> your expectations. Instead I suggest to process XmlSchema
> object
> > >>>>>>> >> only
> > >>>>>>> >> one-time during the deployment after the AxisService creation
> and
> > >>>>>>> >> store result of XmlSchema processing with AxisService object
> as
> > >>>>>>> >> a
> > >>>>>>> >> light-weight object ( Let's say a map) and use this light
> weight
> > >>>>>>> >> map
> > >>>>>>> >> with response. In this approach overhead if relatively low.
> > >>>>>>> >
> > >>>>>>> >
> > >>>>>>> > This is a POC level approach to convert xml stream to json
> using
> > >>>>>>> > xml schema.
> > >>>>>>> > For an example there are some techniques like
> > >>>>>>> > bagger fish and others to address this problem. But those
> things
> > >>>>>>> > complicated
> > >>>>>>> > the produced json stream.
> > >>>>>>> >
> > >>>>>>> > Once it proves at the POC level we can think of improving
> > >>>>>>> > performance.
> > >>>>>>> >
> > >>>>>>> > If you look at the current axis2 json support does that
> properly
> > >>>>>>> > address the
> > >>>>>>> > problem shameera mentioned earlier? i.e serialising the
> > >>>>>>> > array elements when there is on element. And also can that be
> used
> > >>>>>>> > with some
> > >>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
> > >>>>>>> > Idea here to develop a technique to address those problems
> properly
> > >>>>>>> > while
> > >>>>>>> > converting an xml stream to json stream.
> > >>>>>>>
> > >>>>>>> I don't have any issue with the technique proposed here (i.e.
> > >>>>>>> processing XMLSchema to identify structure ). My only concern is
> why
> > >>>>>>> we should process XMLSchema with every response because in each
> and
> > >>>>>>> every response we end up getting same result, why shouldn't we
> > >>>>>>> process
> > >>>>>>> XMLSchema only one time and reuse results ? May be as last step
> of
> > >>>>>>> deployment or with very first response processing ?
> > >>>>>>>
> > >>>>>>> Of course we can keep it as future improvement for the moment but
> > >>>>>>> Axis2 having good history of forgetting those improvement after
> > >>>>>>> initial development every time when I dig into code I found
> couple of
> > >>>>>>> such TODOs.
> > >>>>>>
> > >>>>>>
> > >>>>>> :) I am not telling to do it after GSoc project. But first we
> need to
> > >>>>>> have a proper POC.
> > >>>>>>
> > >>>>>> thanks,
> > >>>>>> Amila.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Thanks !
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> >
> > >>>>>>> > thanks,
> > >>>>>>> > Amila.
> > >>>>>>> >
> > >>>>>>> >>
> > >>>>>>> >>
> > >>>>>>> >> Thanks !
> > >>>>>>> >>
> > >>>>>>> >> > use a queue to keep the schema structure. At the moment i
> have
> > >>>>>>> >> > only
> > >>>>>>> >> > implemented
> > >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
> > >>>>>>> >> > text) ,
> > >>>>>>> >> > writeEndElement()
> > >>>>>>> >> > and writeEndDocument() functions. I am processing the queue
> > >>>>>>> >> > which has
> > >>>>>>> >> > the schema structure and put it to another stack to
> identify the
> > >>>>>>> >> > scope
> > >>>>>>> >> > of
> > >>>>>>> >> > end elements.
> > >>>>>>> >> >
> > >>>>>>> >> > I am now working on, improving this to support xml which has
> > >>>>>>> >> > complex
> > >>>>>>> >> > type
> > >>>>>>> >> > elements
> > >>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean
> while.
> > >>>>>>> >> >
> > >>>>>>> >> > Thanks,
> > >>>>>>> >> > Shameera.
> > >>>>>>> >> >
> > >>>>>>> >> > --
> > >>>>>>> >> > Shameera Rathnayaka
> > >>>>>>> >> > Undergraduate
> > >>>>>>> >> > Department of Computer Science and Engineering
> > >>>>>>> >> > University of Moratuwa.
> > >>>>>>> >> > Sri Lanka.
> > >>>>>>> >> >
> > >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>> >> >
> > >>>>>>> >>
> > >>>>>>> >>
> > >>>>>>> >>
> > >>>>>>> >> --
> > >>>>>>> >> Sagara Gunathunga
> > >>>>>>> >>
> > >>>>>>> >> Blog - http://ssagara.blogspot.com
> > >>>>>>> >> Web - http://people.apache.org/~sagara/
> > >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
> > >>>>>>> >>
> > >>>>>>> >>
> > >>>>>>> >>
> ---------------------------------------------------------------------
> > >>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > >>>>>>> >> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> > >>>>>>> >>
> > >>>>>>> >
> > >>>>>>> >
> > >>>>>>> >
> > >>>>>>> > --
> > >>>>>>> > Amila Suriarachchi
> > >>>>>>> > WSO2 Inc.
> > >>>>>>> > blog: http://amilachinthaka.blogspot.com/
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Sagara Gunathunga
> > >>>>>>>
> > >>>>>>> Blog - http://ssagara.blogspot.com
> > >>>>>>> Web - http://people.apache.org/~sagara/
> > >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
> > >>>>>>>
> > >>>>>>>
> ---------------------------------------------------------------------
> > >>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > >>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Amila Suriarachchi
> > >>>>>> WSO2 Inc.
> > >>>>>> blog: http://amilachinthaka.blogspot.com/
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Shameera Rathnayaka
> > >>>>> Undergraduate
> > >>>>> Department of Computer Science and Engineering
> > >>>>> University of Moratuwa.
> > >>>>> Sri Lanka.
> > >>>>>
> > >>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Amila Suriarachchi
> > >>>> WSO2 Inc.
> > >>>> blog: http://amilachinthaka.blogspot.com/
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Shameera Rathnayaka
> > >>> Undergraduate
> > >>> Department of Computer Science and Engineering
> > >>> University of Moratuwa.
> > >>> Sri Lanka.
> > >>>
> > >>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Shameera Rathnayaka
> > >> Undergraduate
> > >> Department of Computer Science and Engineering
> > >> University of Moratuwa.
> > >> Sri Lanka.
> > >>
> > >> Blog : http://shameerarathnayaka.blogspot.com/
> > >>
> > >
> > >
> > >
> > > --
> > > Shameera Rathnayaka
> > > Undergraduate
> > > Department of Computer Science and Engineering
> > > University of Moratuwa.
> > > Sri Lanka.
> > >
> > > Blog : http://shameerarathnayaka.blogspot.com/
> > >
> >
> >
> >
> > --
> > Sagara Gunathunga
> >
> > Blog - http://ssagara.blogspot.com
> > Web - http://people.apache.org/~sagara/
> > LinkedIn - http://www.linkedin.com/in/ssagara
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
>
--
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.
Blog : http://shameerarathnayaka.blogspot.com/
RE: [GSoC 2012] XML to JSON convention using XMLStreamWriter API
and Schemas with google-gson
Posted by Martin Gainty <mg...@hotmail.com>.
It seems we lose alot of meta data by casting off the SOAP 1.1 superstructure a partial list of what we lose would include:
ability to configure to differing styles (JIBX-RI vs ADB vs XMLBeans)
ability to adhere to different protocols (SMTP and or FTP and or HTTP 1.1)
Ability to incorporate differing XSD's
Parser for both the Input and Output Message Receivers (MEP) goes away
I would like to know what advantages people see with using strictly JSON over a SOAP formatted sender and receiver
thanks!
Martin
______________________________________________
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni.
> From: sagara.gunathunga@gmail.com
> Date: Sat, 11 Aug 2012 19:38:16 +0530
> Subject: Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and Schemas with google-gson
> To: java-dev@axis.apache.org
>
> It seems your 1st approach provides much better performance than 2nd
> approach, can you incorporate improvements that I mentioned here[1] ?
> I believe applying those modification will solve performance issue
> with 2nd approach.
>
> [1] - http://markmail.org/message/xyfxzn2clc3r5zfg
>
> Thanks !
>
> On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
> <sh...@gmail.com> wrote:
> > Hi devs,
> >
> > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON
> > first approach(pure json) and 4. JSON second
> > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
> > summarize the results i got.
> > I used same POJO service and send the exact same request in relevent
> > convention.
> >
> > Environment:
> > OS : Ubuntu 11.04
> > JDK 1.6_30
> > memory 4 GB
> >
> > SOAP JSON badgerfish
> > JSON 1st approach JSON 2nd approach
> >
> > Concurrency Level: 50 50
> > 50 50
> > Time taken for tests: 243.13 sec 287.18 sec
> > 118.99 200.0
> > Complete requests: 500000 500000
> > 500000 500000
> > Total transferred: 2.2 GB 34.42 GB
> > 0.61 GB 0.63 GB
> > Requests per second: 2056.53 #/sec 1741.04 #/sec 4201.69
> > #/sec 2499.91 #/sec
> > Time per request: 24.3 ms 28.7 ms
> > 11.90 ms 20.00 ms
> > Time per request:(mean) 0.486 ms 0.544 ms 0.238
> > ms 0.4 ms
> >
> > Thanks,
> > Shameera.
> >
> >
> > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
> > <sh...@gmail.com> wrote:
> >>
> >> HI devs,
> >>
> >> I finished the implementation and did a performance test with existing
> >> SOAP and implemented JSON. Here is the summary of the test results.
> >> For this i used same service and send the same request in both SOAP and
> >> JSON ways, size of JSON message is lower than SOAP message but requests are
> >> exactly same.
> >>
> >> Environment :
> >> OS - Ubuntu 11.10
> >> Axis2 1.7(current trunk)
> >> Tool - Java benchmark
> >>
> >> num of requests : 50000
> >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
> >> Time taken for test : with SOAP 178 sec and with JSON 140 sec
> >> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000
> >> bytes
> >> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON
> >> 3,568.89 [#/sec] (mean)
> >>
> >> will do a performance test with concurrency level 50 and num of requests
> >> 500000 and give the summary of results.
> >>
> >> Thanks,
> >> Shameera.
> >>
> >>
> >>
> >>
> >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
> >> <sh...@gmail.com> wrote:
> >>>
> >>> HI devs,
> >>>
> >>> I have implemented XMLStreamWriter as well as XMLStreamReader to convert
> >>> XML <----> JSON with XMLSchema,
> >>> Basically it works fine, but still need few test to be done after that I
> >>> am willing to write performance test with implemented JSON support with
> >>> existing SOAP support.
> >>>
> >>> Thanks,
> >>> Shameera.
> >>>
> >>>
> >>>
> >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
> >>> <am...@gmail.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
> >>>> <sh...@gmail.com> wrote:
> >>>>>
> >>>>> Hi devs,
> >>>>>
> >>>>> I implemented this feature completely to convert XML to JSON with the
> >>>>> help of XmlSchema of that XML document. I tested this with different
> >>>>> Schemas, it is working fine. I implemented this as a separate project, so i
> >>>>> will integrate this with axis2 trunk and try with actual scenario.
> >>>>
> >>>>
> >>>> Can you use this technique at the reader level as well to support
> >>>> namespaces?
> >>>>
> >>>> Then do a performance comparison using a POJO service with the exiting
> >>>> json support and based the new gson based XMLstream API. According to the
> >>>> result we need to optimise the later to have better performance.
> >>>>
> >>>> thanks,
> >>>> Amila.
> >>>>
> >>>>>
> >>>>>
> >>>>> Yes, This may be a standard way to convert XML to JSON conversion as
> >>>>> other implementations don't convert JSON message correctly when it has one
> >>>>> value array, unless implementation supports special XML structure where
> >>>>> Json-lib does.
> >>>>>
> >>>>> Definitely we can improve the performance of this implementation. May
> >>>>> be in the way that Sagara had suggested in his reply or in another way.
> >>>>>
> >>>>> Thanks,
> >>>>> Shameera.
> >>>>>
> >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
> >>>>> <am...@gmail.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
> >>>>>> <sa...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
> >>>>>>> <am...@gmail.com> wrote:
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
> >>>>>>> > <sa...@gmail.com> wrote:
> >>>>>>> >>
> >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
> >>>>>>> >> <sh...@gmail.com> wrote:
> >>>>>>> >> > Hi devs,
> >>>>>>> >> >
> >>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I
> >>>>>>> >> > have
> >>>>>>> >> > implemented
> >>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and gson
> >>>>>>> >> > together
> >>>>>>> >> > to convert XML ---> JSON. I have made my progress up to support
> >>>>>>> >> > xml
> >>>>>>> >> > which
> >>>>>>> >> > don't have complex type elements with maxOccur >1, but may have
> >>>>>>> >> > simple
> >>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point
> >>>>>>> >> > of view,
> >>>>>>> >> > my
> >>>>>>> >> > present implementation supports XML --> JSON where there is not
> >>>>>>> >> > any JSON
> >>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
> >>>>>>> >> >
> >>>>>>> >> > Here is a summary of the implementation.
> >>>>>>> >> >
> >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
> >>>>>>> >> > constructor
> >>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
> >>>>>>> >> > XmlSchema and
> >>>>>>> >>
> >>>>>>> >> Are you plan to process XmlSchema object with every response ? If
> >>>>>>> >> so
> >>>>>>> >> processing XmlSchema object will add some overhead and result
> >>>>>>> >> into
> >>>>>>> >> performance. Axis2 already has JSON support main objective of
> >>>>>>> >> this
> >>>>>>> >> project is to provide a high performance JSON implementation,
> >>>>>>> >> processing XmlSchema object with every response does not help to
> >>>>>>> >> meet
> >>>>>>> >> your expectations. Instead I suggest to process XmlSchema object
> >>>>>>> >> only
> >>>>>>> >> one-time during the deployment after the AxisService creation and
> >>>>>>> >> store result of XmlSchema processing with AxisService object as
> >>>>>>> >> a
> >>>>>>> >> light-weight object ( Let's say a map) and use this light weight
> >>>>>>> >> map
> >>>>>>> >> with response. In this approach overhead if relatively low.
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > This is a POC level approach to convert xml stream to json using
> >>>>>>> > xml schema.
> >>>>>>> > For an example there are some techniques like
> >>>>>>> > bagger fish and others to address this problem. But those things
> >>>>>>> > complicated
> >>>>>>> > the produced json stream.
> >>>>>>> >
> >>>>>>> > Once it proves at the POC level we can think of improving
> >>>>>>> > performance.
> >>>>>>> >
> >>>>>>> > If you look at the current axis2 json support does that properly
> >>>>>>> > address the
> >>>>>>> > problem shameera mentioned earlier? i.e serialising the
> >>>>>>> > array elements when there is on element. And also can that be used
> >>>>>>> > with some
> >>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
> >>>>>>> > Idea here to develop a technique to address those problems properly
> >>>>>>> > while
> >>>>>>> > converting an xml stream to json stream.
> >>>>>>>
> >>>>>>> I don't have any issue with the technique proposed here (i.e.
> >>>>>>> processing XMLSchema to identify structure ). My only concern is why
> >>>>>>> we should process XMLSchema with every response because in each and
> >>>>>>> every response we end up getting same result, why shouldn't we
> >>>>>>> process
> >>>>>>> XMLSchema only one time and reuse results ? May be as last step of
> >>>>>>> deployment or with very first response processing ?
> >>>>>>>
> >>>>>>> Of course we can keep it as future improvement for the moment but
> >>>>>>> Axis2 having good history of forgetting those improvement after
> >>>>>>> initial development every time when I dig into code I found couple of
> >>>>>>> such TODOs.
> >>>>>>
> >>>>>>
> >>>>>> :) I am not telling to do it after GSoc project. But first we need to
> >>>>>> have a proper POC.
> >>>>>>
> >>>>>> thanks,
> >>>>>> Amila.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks !
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> >
> >>>>>>> > thanks,
> >>>>>>> > Amila.
> >>>>>>> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> Thanks !
> >>>>>>> >>
> >>>>>>> >> > use a queue to keep the schema structure. At the moment i have
> >>>>>>> >> > only
> >>>>>>> >> > implemented
> >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
> >>>>>>> >> > text) ,
> >>>>>>> >> > writeEndElement()
> >>>>>>> >> > and writeEndDocument() functions. I am processing the queue
> >>>>>>> >> > which has
> >>>>>>> >> > the schema structure and put it to another stack to identify the
> >>>>>>> >> > scope
> >>>>>>> >> > of
> >>>>>>> >> > end elements.
> >>>>>>> >> >
> >>>>>>> >> > I am now working on, improving this to support xml which has
> >>>>>>> >> > complex
> >>>>>>> >> > type
> >>>>>>> >> > elements
> >>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean while.
> >>>>>>> >> >
> >>>>>>> >> > Thanks,
> >>>>>>> >> > Shameera.
> >>>>>>> >> >
> >>>>>>> >> > --
> >>>>>>> >> > Shameera Rathnayaka
> >>>>>>> >> > Undergraduate
> >>>>>>> >> > Department of Computer Science and Engineering
> >>>>>>> >> > University of Moratuwa.
> >>>>>>> >> > Sri Lanka.
> >>>>>>> >> >
> >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>> >> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> --
> >>>>>>> >> Sagara Gunathunga
> >>>>>>> >>
> >>>>>>> >> Blog - http://ssagara.blogspot.com
> >>>>>>> >> Web - http://people.apache.org/~sagara/
> >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> ---------------------------------------------------------------------
> >>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>> >>
> >>>>>>> >
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > --
> >>>>>>> > Amila Suriarachchi
> >>>>>>> > WSO2 Inc.
> >>>>>>> > blog: http://amilachinthaka.blogspot.com/
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Sagara Gunathunga
> >>>>>>>
> >>>>>>> Blog - http://ssagara.blogspot.com
> >>>>>>> Web - http://people.apache.org/~sagara/
> >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>>
> >>>>>>> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Amila Suriarachchi
> >>>>>> WSO2 Inc.
> >>>>>> blog: http://amilachinthaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Shameera Rathnayaka
> >>>>> Undergraduate
> >>>>> Department of Computer Science and Engineering
> >>>>> University of Moratuwa.
> >>>>> Sri Lanka.
> >>>>>
> >>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Amila Suriarachchi
> >>>> WSO2 Inc.
> >>>> blog: http://amilachinthaka.blogspot.com/
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Shameera Rathnayaka
> >>> Undergraduate
> >>> Department of Computer Science and Engineering
> >>> University of Moratuwa.
> >>> Sri Lanka.
> >>>
> >>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>
> >>
> >>
> >> --
> >> Shameera Rathnayaka
> >> Undergraduate
> >> Department of Computer Science and Engineering
> >> University of Moratuwa.
> >> Sri Lanka.
> >>
> >> Blog : http://shameerarathnayaka.blogspot.com/
> >>
> >
> >
> >
> > --
> > Shameera Rathnayaka
> > Undergraduate
> > Department of Computer Science and Engineering
> > University of Moratuwa.
> > Sri Lanka.
> >
> > Blog : http://shameerarathnayaka.blogspot.com/
> >
>
>
>
> --
> Sagara Gunathunga
>
> Blog - http://ssagara.blogspot.com
> Web - http://people.apache.org/~sagara/
> LinkedIn - http://www.linkedin.com/in/ssagara
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
Posted by Sagara Gunathunga <sa...@gmail.com>.
It seems your 1st approach provides much better performance than 2nd
approach, can you incorporate improvements that I mentioned here[1] ?
I believe applying those modification will solve performance issue
with 2nd approach.
[1] - http://markmail.org/message/xyfxzn2clc3r5zfg
Thanks !
On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
<sh...@gmail.com> wrote:
> Hi devs,
>
> I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON
> first approach(pure json) and 4. JSON second
> approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
> summarize the results i got.
> I used same POJO service and send the exact same request in relevent
> convention.
>
> Environment:
> OS : Ubuntu 11.04
> JDK 1.6_30
> memory 4 GB
>
> SOAP JSON badgerfish
> JSON 1st approach JSON 2nd approach
>
> Concurrency Level: 50 50
> 50 50
> Time taken for tests: 243.13 sec 287.18 sec
> 118.99 200.0
> Complete requests: 500000 500000
> 500000 500000
> Total transferred: 2.2 GB 34.42 GB
> 0.61 GB 0.63 GB
> Requests per second: 2056.53 #/sec 1741.04 #/sec 4201.69
> #/sec 2499.91 #/sec
> Time per request: 24.3 ms 28.7 ms
> 11.90 ms 20.00 ms
> Time per request:(mean) 0.486 ms 0.544 ms 0.238
> ms 0.4 ms
>
> Thanks,
> Shameera.
>
>
> On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
> <sh...@gmail.com> wrote:
>>
>> HI devs,
>>
>> I finished the implementation and did a performance test with existing
>> SOAP and implemented JSON. Here is the summary of the test results.
>> For this i used same service and send the same request in both SOAP and
>> JSON ways, size of JSON message is lower than SOAP message but requests are
>> exactly same.
>>
>> Environment :
>> OS - Ubuntu 11.10
>> Axis2 1.7(current trunk)
>> Tool - Java benchmark
>>
>> num of requests : 50000
>> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
>> Time taken for test : with SOAP 178 sec and with JSON 140 sec
>> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000
>> bytes
>> Requests per second: with SOAP 2,801.76 [#/sec] (mean) and with JSON
>> 3,568.89 [#/sec] (mean)
>>
>> will do a performance test with concurrency level 50 and num of requests
>> 500000 and give the summary of results.
>>
>> Thanks,
>> Shameera.
>>
>>
>>
>>
>> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
>> <sh...@gmail.com> wrote:
>>>
>>> HI devs,
>>>
>>> I have implemented XMLStreamWriter as well as XMLStreamReader to convert
>>> XML <----> JSON with XMLSchema,
>>> Basically it works fine, but still need few test to be done after that I
>>> am willing to write performance test with implemented JSON support with
>>> existing SOAP support.
>>>
>>> Thanks,
>>> Shameera.
>>>
>>>
>>>
>>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
>>> <am...@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
>>>> <sh...@gmail.com> wrote:
>>>>>
>>>>> Hi devs,
>>>>>
>>>>> I implemented this feature completely to convert XML to JSON with the
>>>>> help of XmlSchema of that XML document. I tested this with different
>>>>> Schemas, it is working fine. I implemented this as a separate project, so i
>>>>> will integrate this with axis2 trunk and try with actual scenario.
>>>>
>>>>
>>>> Can you use this technique at the reader level as well to support
>>>> namespaces?
>>>>
>>>> Then do a performance comparison using a POJO service with the exiting
>>>> json support and based the new gson based XMLstream API. According to the
>>>> result we need to optimise the later to have better performance.
>>>>
>>>> thanks,
>>>> Amila.
>>>>
>>>>>
>>>>>
>>>>> Yes, This may be a standard way to convert XML to JSON conversion as
>>>>> other implementations don't convert JSON message correctly when it has one
>>>>> value array, unless implementation supports special XML structure where
>>>>> Json-lib does.
>>>>>
>>>>> Definitely we can improve the performance of this implementation. May
>>>>> be in the way that Sagara had suggested in his reply or in another way.
>>>>>
>>>>> Thanks,
>>>>> Shameera.
>>>>>
>>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
>>>>> <am...@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
>>>>>> <sa...@gmail.com> wrote:
>>>>>>>
>>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
>>>>>>> <am...@gmail.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
>>>>>>> > <sa...@gmail.com> wrote:
>>>>>>> >>
>>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
>>>>>>> >> <sh...@gmail.com> wrote:
>>>>>>> >> > Hi devs,
>>>>>>> >> >
>>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I
>>>>>>> >> > have
>>>>>>> >> > implemented
>>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and gson
>>>>>>> >> > together
>>>>>>> >> > to convert XML ---> JSON. I have made my progress up to support
>>>>>>> >> > xml
>>>>>>> >> > which
>>>>>>> >> > don't have complex type elements with maxOccur >1, but may have
>>>>>>> >> > simple
>>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point
>>>>>>> >> > of view,
>>>>>>> >> > my
>>>>>>> >> > present implementation supports XML --> JSON where there is not
>>>>>>> >> > any JSON
>>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
>>>>>>> >> >
>>>>>>> >> > Here is a summary of the implementation.
>>>>>>> >> >
>>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
>>>>>>> >> > constructor
>>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
>>>>>>> >> > XmlSchema and
>>>>>>> >>
>>>>>>> >> Are you plan to process XmlSchema object with every response ? If
>>>>>>> >> so
>>>>>>> >> processing XmlSchema object will add some overhead and result
>>>>>>> >> into
>>>>>>> >> performance. Axis2 already has JSON support main objective of
>>>>>>> >> this
>>>>>>> >> project is to provide a high performance JSON implementation,
>>>>>>> >> processing XmlSchema object with every response does not help to
>>>>>>> >> meet
>>>>>>> >> your expectations. Instead I suggest to process XmlSchema object
>>>>>>> >> only
>>>>>>> >> one-time during the deployment after the AxisService creation and
>>>>>>> >> store result of XmlSchema processing with AxisService object as
>>>>>>> >> a
>>>>>>> >> light-weight object ( Let's say a map) and use this light weight
>>>>>>> >> map
>>>>>>> >> with response. In this approach overhead if relatively low.
>>>>>>> >
>>>>>>> >
>>>>>>> > This is a POC level approach to convert xml stream to json using
>>>>>>> > xml schema.
>>>>>>> > For an example there are some techniques like
>>>>>>> > bagger fish and others to address this problem. But those things
>>>>>>> > complicated
>>>>>>> > the produced json stream.
>>>>>>> >
>>>>>>> > Once it proves at the POC level we can think of improving
>>>>>>> > performance.
>>>>>>> >
>>>>>>> > If you look at the current axis2 json support does that properly
>>>>>>> > address the
>>>>>>> > problem shameera mentioned earlier? i.e serialising the
>>>>>>> > array elements when there is on element. And also can that be used
>>>>>>> > with some
>>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
>>>>>>> > Idea here to develop a technique to address those problems properly
>>>>>>> > while
>>>>>>> > converting an xml stream to json stream.
>>>>>>>
>>>>>>> I don't have any issue with the technique proposed here (i.e.
>>>>>>> processing XMLSchema to identify structure ). My only concern is why
>>>>>>> we should process XMLSchema with every response because in each and
>>>>>>> every response we end up getting same result, why shouldn't we
>>>>>>> process
>>>>>>> XMLSchema only one time and reuse results ? May be as last step of
>>>>>>> deployment or with very first response processing ?
>>>>>>>
>>>>>>> Of course we can keep it as future improvement for the moment but
>>>>>>> Axis2 having good history of forgetting those improvement after
>>>>>>> initial development every time when I dig into code I found couple of
>>>>>>> such TODOs.
>>>>>>
>>>>>>
>>>>>> :) I am not telling to do it after GSoc project. But first we need to
>>>>>> have a proper POC.
>>>>>>
>>>>>> thanks,
>>>>>> Amila.
>>>>>>>
>>>>>>>
>>>>>>> Thanks !
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> >
>>>>>>> > thanks,
>>>>>>> > Amila.
>>>>>>> >
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> Thanks !
>>>>>>> >>
>>>>>>> >> > use a queue to keep the schema structure. At the moment i have
>>>>>>> >> > only
>>>>>>> >> > implemented
>>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
>>>>>>> >> > text) ,
>>>>>>> >> > writeEndElement()
>>>>>>> >> > and writeEndDocument() functions. I am processing the queue
>>>>>>> >> > which has
>>>>>>> >> > the schema structure and put it to another stack to identify the
>>>>>>> >> > scope
>>>>>>> >> > of
>>>>>>> >> > end elements.
>>>>>>> >> >
>>>>>>> >> > I am now working on, improving this to support xml which has
>>>>>>> >> > complex
>>>>>>> >> > type
>>>>>>> >> > elements
>>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean while.
>>>>>>> >> >
>>>>>>> >> > Thanks,
>>>>>>> >> > Shameera.
>>>>>>> >> >
>>>>>>> >> > --
>>>>>>> >> > Shameera Rathnayaka
>>>>>>> >> > Undergraduate
>>>>>>> >> > Department of Computer Science and Engineering
>>>>>>> >> > University of Moratuwa.
>>>>>>> >> > Sri Lanka.
>>>>>>> >> >
>>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>> >> >
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> --
>>>>>>> >> Sagara Gunathunga
>>>>>>> >>
>>>>>>> >> Blog - http://ssagara.blogspot.com
>>>>>>> >> Web - http://people.apache.org/~sagara/
>>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> ---------------------------------------------------------------------
>>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>> >>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > --
>>>>>>> > Amila Suriarachchi
>>>>>>> > WSO2 Inc.
>>>>>>> > blog: http://amilachinthaka.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sagara Gunathunga
>>>>>>>
>>>>>>> Blog - http://ssagara.blogspot.com
>>>>>>> Web - http://people.apache.org/~sagara/
>>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Amila Suriarachchi
>>>>>> WSO2 Inc.
>>>>>> blog: http://amilachinthaka.blogspot.com/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Shameera Rathnayaka
>>>>> Undergraduate
>>>>> Department of Computer Science and Engineering
>>>>> University of Moratuwa.
>>>>> Sri Lanka.
>>>>>
>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Amila Suriarachchi
>>>> WSO2 Inc.
>>>> blog: http://amilachinthaka.blogspot.com/
>>>
>>>
>>>
>>>
>>> --
>>> Shameera Rathnayaka
>>> Undergraduate
>>> Department of Computer Science and Engineering
>>> University of Moratuwa.
>>> Sri Lanka.
>>>
>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>
>>
>>
>>
>> --
>> Shameera Rathnayaka
>> Undergraduate
>> Department of Computer Science and Engineering
>> University of Moratuwa.
>> Sri Lanka.
>>
>> Blog : http://shameerarathnayaka.blogspot.com/
>>
>
>
>
> --
> Shameera Rathnayaka
> Undergraduate
> Department of Computer Science and Engineering
> University of Moratuwa.
> Sri Lanka.
>
> Blog : http://shameerarathnayaka.blogspot.com/
>
--
Sagara Gunathunga
Blog - http://ssagara.blogspot.com
Web - http://people.apache.org/~sagara/
LinkedIn - http://www.linkedin.com/in/ssagara
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org
Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and
Schemas with google-gson
Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi devs,
I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON
first approach(pure json) and 4. JSON second
approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
summarize the results i got.
I used same POJO service and send the exact same request in relevent
convention.
Environment:
OS : Ubuntu 11.04
JDK 1.6_30
memory 4 GB
* SOAP JSON
badgerfish JSON 1st approach JSON 2nd approach
Concurrency Level: *50
50 50 50
*Time taken for tests:* 243.13 sec 287.18 sec
118.99 200.0
*Complete requests: *500000 500000*
*500000 500000 *
Total transferred:* 2.2 GB 34.42
GB 0.61 GB 0.63 GB
*Requests per second: *2056.53* *#/sec 1741.04 #/sec
4201.69 #/sec 2499.91 #/sec
*Time per request: *24.3 ms 28.7 ms
11.90 ms 20.00 ms *
Time per request:(mean) *0.486* *ms 0.544 ms
0.238 ms 0.4 ms
Thanks,
Shameera.
On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka <shameerainfo@gmail.com
> wrote:
> HI devs,
>
> I finished the implementation and did a performance test with existing
> SOAP and implemented JSON. Here is the summary of the test results.
> For this i used same service and send the same request in both SOAP and
> JSON ways, size of JSON message is lower than SOAP message but requests are
> exactly same.
>
> Environment :
> OS - Ubuntu 11.10
> Axis2 1.7(current trunk)
> Tool - Java benchmark
>
> *num of requests* : 50000
> *concurrency level*: 10 (Total num of requests are 50000*10 = 500000 )
> *Time taken for test* : with SOAP *178* sec and with JSON *140* sec
> *Total transferred* : with SOAP *2371000000* bytes and with JSON *
> 674000000* bytes
> *Requests per second*: with SOAP * 2,801.76* [#/sec] (mean) and with
> JSON * 3,568.89* [#/sec] (mean)
>
> will do a performance test with concurrency level 50 and num of requests
> 500000 and give the summary of results.
>
> Thanks,
> Shameera.
>
>
>
>
> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka <
> shameerainfo@gmail.com> wrote:
>
>> HI devs,
>>
>> I have implemented XMLStreamWriter as well as XMLStreamReader to convert
>> XML <----> JSON with XMLSchema,
>> Basically it works fine, but still need few test to be done after that I
>> am willing to write performance test with implemented JSON support with
>> existing SOAP support.
>>
>> Thanks,
>> Shameera.
>>
>>
>>
>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi <
>> amilasuriarachchi@gmail.com> wrote:
>>
>>>
>>>
>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com> wrote:
>>>
>>>> Hi devs,
>>>>
>>>> I implemented this feature completely to convert XML to JSON with the
>>>> help of XmlSchema of that XML document. I tested this with different
>>>> Schemas, it is working fine. I implemented this as a separate project, so i
>>>> will integrate this with axis2 trunk and try with actual scenario.
>>>>
>>>
>>> Can you use this technique at the reader level as well to support
>>> namespaces?
>>>
>>> Then do a performance comparison using a POJO service with the exiting
>>> json support and based the new gson based XMLstream API. According to the
>>> result we need to optimise the later to have better performance.
>>>
>>> thanks,
>>> Amila.
>>>
>>>
>>>>
>>>> Yes, This may be a standard way to convert XML to JSON conversion as
>>>> other implementations don't convert JSON message correctly when it has one
>>>> value array, unless implementation supports special XML structure where
>>>> Json-lib does.
>>>>
>>>> Definitely we can improve the performance of this implementation. May
>>>> be in the way that Sagara had suggested in his reply or in another way.
>>>>
>>>> Thanks,
>>>> Shameera.
>>>>
>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi <
>>>> amilasuriarachchi@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga <
>>>>> sagara.gunathunga@gmail.com> wrote:
>>>>>
>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
>>>>>> <am...@gmail.com> wrote:
>>>>>> >
>>>>>> >
>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
>>>>>> > <sa...@gmail.com> wrote:
>>>>>> >>
>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
>>>>>> >> <sh...@gmail.com> wrote:
>>>>>> >> > Hi devs,
>>>>>> >> >
>>>>>> >> > As Amila(Project mentor) suggested in previous thread[0] , I have
>>>>>> >> > implemented
>>>>>> >> > XMLStreamWriter API to use schema of the processing xml and gson
>>>>>> >> > together
>>>>>> >> > to convert XML ---> JSON. I have made my progress up to support
>>>>>> xml
>>>>>> >> > which
>>>>>> >> > don't have complex type elements with maxOccur >1, but may have
>>>>>> simple
>>>>>> >> > type elements with maxOccur >=1 . That means In the JSON point
>>>>>> of view,
>>>>>> >> > my
>>>>>> >> > present implementation supports XML --> JSON where there is not
>>>>>> any JSON
>>>>>> >> > Arrays which has JSON objects in the converted JSON String.
>>>>>> >> >
>>>>>> >> > Here is a summary of the implementation.
>>>>>> >> >
>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter has a
>>>>>> constructor
>>>>>> >> > to get JsonWriter and XmlSchema object. Then it process
>>>>>> XmlSchema and
>>>>>> >>
>>>>>> >> Are you plan to process XmlSchema object with every response ? If
>>>>>> so
>>>>>> >> processing XmlSchema object will add some overhead and result into
>>>>>> >> performance. Axis2 already has JSON support main objective of this
>>>>>> >> project is to provide a high performance JSON implementation,
>>>>>> >> processing XmlSchema object with every response does not help to
>>>>>> meet
>>>>>> >> your expectations. Instead I suggest to process XmlSchema object
>>>>>> only
>>>>>> >> one-time during the deployment after the AxisService creation and
>>>>>> >> store result of XmlSchema processing with AxisService object as
>>>>>> a
>>>>>> >> light-weight object ( Let's say a map) and use this light weight
>>>>>> map
>>>>>> >> with response. In this approach overhead if relatively low.
>>>>>> >
>>>>>> >
>>>>>> > This is a POC level approach to convert xml stream to json using
>>>>>> xml schema.
>>>>>> > For an example there are some techniques like
>>>>>> > bagger fish and others to address this problem. But those things
>>>>>> complicated
>>>>>> > the produced json stream.
>>>>>> >
>>>>>> > Once it proves at the POC level we can think of improving
>>>>>> performance.
>>>>>> >
>>>>>> > If you look at the current axis2 json support does that properly
>>>>>> address the
>>>>>> > problem shameera mentioned earlier? i.e serialising the
>>>>>> > array elements when there is on element. And also can that be used
>>>>>> with some
>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
>>>>>> > Idea here to develop a technique to address those problems properly
>>>>>> while
>>>>>> > converting an xml stream to json stream.
>>>>>>
>>>>>> I don't have any issue with the technique proposed here (i.e.
>>>>>> processing XMLSchema to identify structure ). My only concern is why
>>>>>> we should process XMLSchema with every response because in each and
>>>>>> every response we end up getting same result, why shouldn't we process
>>>>>> XMLSchema only one time and reuse results ? May be as last step of
>>>>>> deployment or with very first response processing ?
>>>>>>
>>>>>> Of course we can keep it as future improvement for the moment but
>>>>>> Axis2 having good history of forgetting those improvement after
>>>>>> initial development every time when I dig into code I found couple of
>>>>>> such TODOs.
>>>>>>
>>>>>
>>>>> :) I am not telling to do it after GSoc project. But first we need to
>>>>> have a proper POC.
>>>>>
>>>>> thanks,
>>>>> Amila.
>>>>>
>>>>>>
>>>>>> Thanks !
>>>>>>
>>>>>>
>>>>>>
>>>>>> >
>>>>>> > thanks,
>>>>>> > Amila.
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >> Thanks !
>>>>>> >>
>>>>>> >> > use a queue to keep the schema structure. At the moment i have
>>>>>> only
>>>>>> >> > implemented
>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
>>>>>> text) ,
>>>>>> >> > writeEndElement()
>>>>>> >> > and writeEndDocument() functions. I am processing the queue
>>>>>> which has
>>>>>> >> > the schema structure and put it to another stack to identify the
>>>>>> scope
>>>>>> >> > of
>>>>>> >> > end elements.
>>>>>> >> >
>>>>>> >> > I am now working on, improving this to support xml which has
>>>>>> complex
>>>>>> >> > type
>>>>>> >> > elements
>>>>>> >> > with maxOccour > 1 . I'll keep updating my progress mean while.
>>>>>> >> >
>>>>>> >> > Thanks,
>>>>>> >> > Shameera.
>>>>>> >> >
>>>>>> >> > --
>>>>>> >> > Shameera Rathnayaka
>>>>>> >> > Undergraduate
>>>>>> >> > Department of Computer Science and Engineering
>>>>>> >> > University of Moratuwa.
>>>>>> >> > Sri Lanka.
>>>>>> >> >
>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
>>>>>> >> >
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> --
>>>>>> >> Sagara Gunathunga
>>>>>> >>
>>>>>> >> Blog - http://ssagara.blogspot.com
>>>>>> >> Web - http://people.apache.org/~sagara/
>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>>> >>
>>>>>> >>
>>>>>> ---------------------------------------------------------------------
>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Amila Suriarachchi
>>>>>> > WSO2 Inc.
>>>>>> > blog: http://amilachinthaka.blogspot.com/
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sagara Gunathunga
>>>>>>
>>>>>> Blog - http://ssagara.blogspot.com
>>>>>> Web - http://people.apache.org/~sagara/
>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Amila Suriarachchi
>>>>> WSO2 Inc.
>>>>> blog: http://amilachinthaka.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Shameera Rathnayaka
>>>> Undergraduate
>>>> Department of Computer Science and Engineering
>>>> University of Moratuwa.
>>>> Sri Lanka.
>>>>
>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>
>>>>
>>>
>>>
>>> --
>>> Amila Suriarachchi
>>> WSO2 Inc.
>>> blog: http://amilachinthaka.blogspot.com/
>>>
>>
>>
>>
>> --
>> Shameera Rathnayaka
>> Undergraduate
>> Department of Computer Science and Engineering
>> University of Moratuwa.
>> Sri Lanka.
>>
>> Blog : http://shameerarathnayaka.blogspot.com/
>>
>>
>
>
> --
> Shameera Rathnayaka
> Undergraduate
> Department of Computer Science and Engineering
> University of Moratuwa.
> Sri Lanka.
>
> Blog : http://shameerarathnayaka.blogspot.com/
>
>
--
Shameera Rathnayaka
Undergraduate
Department of Computer Science and Engineering
University of Moratuwa.
Sri Lanka.
Blog : http://shameerarathnayaka.blogspot.com/