You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shindig.apache.org by Paul Lindner <pl...@hi5.com> on 2008/04/29 03:04:48 UTC
Json-lib instead of json.org
Hi,
I¹ve been crawling through an OutOfMemory dump today and traced a problem
back to the json.org parser.
Apparently this truncated JSON will create an 8 million+ size ArrayList:
mySource java.lang.String
"[{"type":"FETCH_PEOPLE","idSpec":"OWNER","profileDetail":["id","name","thum
bnailUrl"],"sortOrder":"topFriends","filter":"all","first":0,"max":20},"
What do people think about switching over to json-lib
(http://json-lib.sourceforge.net/) instead? It appears that this is already
used by Abdera and some other Apache projects.
In addition to this problem I¹ve already had to remove the code that checks
the Mandatory annotation. That code ends up checking a synchronized
HashMap, which causes severe contention problems under load.
Paul
plindner@hi5.com
Re: Json-lib instead of json.org
Posted by Cassie <do...@google.com>.
ya wanna just checkin the code that removes the mandatory annotation paul?
i'm not sure it does enough good to stay. containers can validate their own
data.
just my 2 cents.
- cassie
On Tue, Apr 29, 2008 at 3:31 AM, Fernando Padilla <fe...@alum.mit.edu> wrote:
> I think that for any dependencies that anyone suggests in the future, they
> should already be hosted on some public mvn repository.
>
> If in doubt, search for available jar files in http://mvnrepository.com
>
> It might not index all repositories, but it indexes at least the large
> default one. So you can quickly see what latest versions are easily
> available to people, etc etc.
>
> Though, it depends on those desired projects to publish their jar files
> into the the large repositories hosted by apache or ibiblio.
>
>
>
> For example, you can get to that dependency listed below, via searching:
>
> I searched for json-lib
>
> http://mvnrepository.com/search.html?query=json-lib
>
> and found net.sf.json-lib, json-lib
>
> http://mvnrepository.com/artifact/net.sf.json-lib/json-lib
>
> and the latest version available was: 2.2.1
>
>
>
>
>
> Paul Lindner wrote:
>
>> Updated info, it was actually an ArrayList of size 86,956,821 occupying
>> 695,654,584 bytes of memory... :)
>>
>> On 4/28/08 6:13 PM, "Kevin Brown" <et...@google.com> wrote:
>>
>> On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
>>>
>>>> What do people think about switching over to json-lib
>>>> (http://json-lib.sourceforge.net/) instead? It appears that this is
>>>> already
>>>> used by Abdera and some other Apache projects.
>>>>
>>>
>>> I wouldn't mind using a different JSON library, but the dependency list
>>> of
>>> this one looked a little scary. Is there a maven repo?
>>>
>>
>> Yes. Appears so:
>>
>> <dependency>
>> <groupId>net.sf.json-lib</groupId>
>> <artifactId>json-lib</artifactId>
>> <version>2.2.1</version>
>> <classifier>jdk15</classifier>
>> </dependency>
>>
>> (This is from the Abdera pom.xml)
>>
>>
>>
Re: Json-lib instead of json.org
Posted by Fernando Padilla <fe...@alum.mit.edu>.
I think that for any dependencies that anyone suggests in the future,
they should already be hosted on some public mvn repository.
If in doubt, search for available jar files in http://mvnrepository.com
It might not index all repositories, but it indexes at least the large
default one. So you can quickly see what latest versions are easily
available to people, etc etc.
Though, it depends on those desired projects to publish their jar files
into the the large repositories hosted by apache or ibiblio.
For example, you can get to that dependency listed below, via searching:
I searched for json-lib
http://mvnrepository.com/search.html?query=json-lib
and found net.sf.json-lib, json-lib
http://mvnrepository.com/artifact/net.sf.json-lib/json-lib
and the latest version available was: 2.2.1
Paul Lindner wrote:
> Updated info, it was actually an ArrayList of size 86,956,821 occupying
> 695,654,584 bytes of memory... :)
>
> On 4/28/08 6:13 PM, "Kevin Brown" <et...@google.com> wrote:
>
>> On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
>>> What do people think about switching over to json-lib
>>> (http://json-lib.sourceforge.net/) instead? It appears that this is
>>> already
>>> used by Abdera and some other Apache projects.
>>
>> I wouldn't mind using a different JSON library, but the dependency list of
>> this one looked a little scary. Is there a maven repo?
>
> Yes. Appears so:
>
> <dependency>
> <groupId>net.sf.json-lib</groupId>
> <artifactId>json-lib</artifactId>
> <version>2.2.1</version>
> <classifier>jdk15</classifier>
> </dependency>
>
> (This is from the Abdera pom.xml)
>
>
Re: Json-lib instead of json.org
Posted by Paul Lindner <pl...@hi5.com>.
Updated info, it was actually an ArrayList of size 86,956,821 occupying
695,654,584 bytes of memory... :)
On 4/28/08 6:13 PM, "Kevin Brown" <et...@google.com> wrote:
> On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
>> What do people think about switching over to json-lib
>> (http://json-lib.sourceforge.net/) instead? It appears that this is
>> already
>> used by Abdera and some other Apache projects.
>
>
> I wouldn't mind using a different JSON library, but the dependency list of
> this one looked a little scary. Is there a maven repo?
Yes. Appears so:
<dependency>
<groupId>net.sf.json-lib</groupId>
<artifactId>json-lib</artifactId>
<version>2.2.1</version>
<classifier>jdk15</classifier>
</dependency>
(This is from the Abdera pom.xml)
Re: Json-lib instead of json.org
Posted by Kevin Brown <et...@google.com>.
On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
> Hi,
>
> I¹ve been crawling through an OutOfMemory dump today and traced a problem
> back to the json.org parser.
>
> Apparently this truncated JSON will create an 8 million+ size ArrayList:
>
> mySource java.lang.String
>
> "[{"type":"FETCH_PEOPLE","idSpec":"OWNER","profileDetail":["id","name","thum
> bnailUrl"],"sortOrder":"topFriends","filter":"all","first":0,"max":20},"
>
> What do people think about switching over to json-lib
> (http://json-lib.sourceforge.net/) instead? It appears that this is
> already
> used by Abdera and some other Apache projects.
I wouldn't mind using a different JSON library, but the dependency list of
this one looked a little scary. Is there a maven repo?
>
>
> In addition to this problem I¹ve already had to remove the code that
> checks
> the Mandatory annotation. That code ends up checking a synchronized
> HashMap, which causes severe contention problems under load.
>
> Paul
> plindner@hi5.com
>
>
RE: Json-lib instead of json.org
Posted by Nick Lothian <nl...@educationau.edu.au>.
If you use this method, can you please namespace it (ie, import it into org.apache.shindig.org.json) so it doesn't clash with other projects which have had to do the same thing (eg, my project)
Although theoretically it shouldn't matter I suppose...
Nick
-----Original Message-----
From: Kevin Brown [mailto:etnu@google.com]
Sent: Friday, 2 May 2008 2:09 AM
To: shindig-dev@incubator.apache.org
Subject: Re: Json-lib instead of json.org
If we don't move to the other json lib, we could always just import the
json.org code into shindig directly. I've seen other projects doing this. I
find it unlikely that json.org itself will ever host an SCM repo, but they
might let someone else take over maintenance of the project if asked.
On Thu, May 1, 2008 at 3:26 AM, Paul Lindner <pl...@hi5.com> wrote:
> It appears that the latest code from json.org fixes this bug. I built a
> new
> json.jar and dropped that into my internal build and all is fine.
>
> Shindig builds with some maven snapshot from late 2007. Sadly it appears
> that this code could use a good source code control system. I'm relying
> on
> zip file timestamps to figure out what's been updated, and when.
>
> Any chance someone can update the maven repo with current code?
>
>
> On 4/29/08 11:06 AM, "Brian Eaton" <be...@google.com> wrote:
>
> > On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
> >> What do people think about switching over to json-lib
> >> (http://json-lib.sourceforge.net/) instead? It appears that this is
> already
> >> used by Abdera and some other Apache projects.
> >
> > Any chance of us contributing fixes back to json.org? You've already
> > proved that you can fix json.org code, and we don''t know that the
> > json-lib code will be better quality. (Both are used inside google,
> > but json.org more widely than json-lib.)
>
>
IMPORTANT: This e-mail, including any attachments, may contain private or confidential information. If you think you may not be the intended recipient, or if you have received this e-mail in error, please contact the sender immediately and delete all copies of this e-mail. If you are not the intended recipient, you must not reproduce any part of this e-mail or disclose its contents to any other party. This email represents the views of the individual sender, which do not necessarily reflect those of education.au limited except where the sender expressly states otherwise. It is your responsibility to scan this email and any files transmitted with it for viruses or any other defects. education.au limited will not be liable for any loss, damage or consequence caused directly or indirectly by this email.
Re: Json-lib instead of json.org
Posted by Kevin Brown <et...@google.com>.
If we don't move to the other json lib, we could always just import the
json.org code into shindig directly. I've seen other projects doing this. I
find it unlikely that json.org itself will ever host an SCM repo, but they
might let someone else take over maintenance of the project if asked.
On Thu, May 1, 2008 at 3:26 AM, Paul Lindner <pl...@hi5.com> wrote:
> It appears that the latest code from json.org fixes this bug. I built a
> new
> json.jar and dropped that into my internal build and all is fine.
>
> Shindig builds with some maven snapshot from late 2007. Sadly it appears
> that this code could use a good source code control system. I'm relying
> on
> zip file timestamps to figure out what's been updated, and when.
>
> Any chance someone can update the maven repo with current code?
>
>
> On 4/29/08 11:06 AM, "Brian Eaton" <be...@google.com> wrote:
>
> > On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
> >> What do people think about switching over to json-lib
> >> (http://json-lib.sourceforge.net/) instead? It appears that this is
> already
> >> used by Abdera and some other Apache projects.
> >
> > Any chance of us contributing fixes back to json.org? You've already
> > proved that you can fix json.org code, and we don''t know that the
> > json-lib code will be better quality. (Both are used inside google,
> > but json.org more widely than json-lib.)
>
>
Re: Json-lib instead of json.org
Posted by Paul Lindner <pl...@hi5.com>.
It appears that the latest code from json.org fixes this bug. I built a new
json.jar and dropped that into my internal build and all is fine.
Shindig builds with some maven snapshot from late 2007. Sadly it appears
that this code could use a good source code control system. I'm relying on
zip file timestamps to figure out what's been updated, and when.
Any chance someone can update the maven repo with current code?
On 4/29/08 11:06 AM, "Brian Eaton" <be...@google.com> wrote:
> On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
>> What do people think about switching over to json-lib
>> (http://json-lib.sourceforge.net/) instead? It appears that this is already
>> used by Abdera and some other Apache projects.
>
> Any chance of us contributing fixes back to json.org? You've already
> proved that you can fix json.org code, and we don''t know that the
> json-lib code will be better quality. (Both are used inside google,
> but json.org more widely than json-lib.)
Re: Json-lib instead of json.org
Posted by Brian Eaton <be...@google.com>.
On Mon, Apr 28, 2008 at 6:04 PM, Paul Lindner <pl...@hi5.com> wrote:
> What do people think about switching over to json-lib
> (http://json-lib.sourceforge.net/) instead? It appears that this is already
> used by Abdera and some other Apache projects.
Any chance of us contributing fixes back to json.org? You've already
proved that you can fix json.org code, and we don''t know that the
json-lib code will be better quality. (Both are used inside google,
but json.org more widely than json-lib.)