invalid json allowed into couch

[Norman Barker <no...@gmail.com>]

Hi,

currently if adding documents to couch using the REST api it is
possible to add a document such as

{
  "prop1": "1",
  "prop1": "2"
}

which only displays partially in Futon (skips first property, "prop1":
"1") due to the unique constraints of JSON object properties.

This is actually a good feature, since I am generating invalid JSON
from an event based API, is there a way to add a normalisation
function to the write document operation  (using Erlang ideally) that
would group these properties,  its gets difficult when the documents
are nested but any thoughts appreciated.

Many thanks,

Norman

Re: invalid json allowed into couch

[Paul Davis <pa...@gmail.com>]

Nitin,

My guess is that if you included repeated fields in any sort of JSON
like spec, no one would ever use it. Every JSON parser I've looked at
with the exception of the Erlang parsers don't allow for repeating
property names.

In terms of the feature staying or not, I wouldn't make any sort of
guarantee on it. If I had my way, we'd revise the RFC to s/MAY/MUST/
be unique and disallow this interpretation at all. Some people already
read the RFC as such anyway because of the "is a subset of JS" line.

So the two big reasons that I wouldn't use it would be:

If Erlang gets JSON bifs, they'd enforce uniqueness (assuming they use
YAJL like my initial implementation).
Almost no tool will ever be able to interpret repeated field data properly.

Ie, I consider this a bug not a feature. Though not one that ever
really causes many issues because almost no tool outside of Erlang
would even allow it.

HTH,
Paul Davis

On Sat, Sep 19, 2009 at 8:05 PM, Nitin Borwankar <ni...@borwankar.com> wrote:
> Is this "feature" guaranteed to remain going forward?
> My project has a lot of data that could use the repeated fields feature
> (multiple authors of a bibliographic item, multiple keywords...)
> but this is like pouring concrete - we can't uproot all of the data if a
> decision is made later to be strictly JS compliant.
> We are currently not using this capability, but it certainly looks
> attractive and am curious about its longevity.
>
> Damien or any other committers - any comments ?
>
> Nitin
>
>
> 37% of all statistics are made up on the spot
> -------------------------------------------------------------------------------------
> Nitin Borwankar
> nborwankar@gmail.com
>
>
> On Sat, Sep 19, 2009 at 4:27 PM, Paul Davis <pa...@gmail.com>wrote:
>
>> Technically, this isn't invalid JSON.
>>
>> Depending on how you interpret the spec, it says that properties only
>> "MAY" be enforced as unique. Some people read the line about "being a
>> subset of JavaScript" which does enforce uniqueness to override this
>> sentence.
>>
>> Either way, the JSON serializer that CouchDB uses doesn't enforce
>> uniqueness so it'll happily accept and repeat back objects with
>> repeated property names. You can check with cURL that your doc
>> actually kept both of those properties.
>>
>> As to having a 'fix' for this, the only thing that I could forsee
>> happening is enforcing the constraint and just erroring out on the
>> initial PUT request. Concatenating repeated fields would be something
>> that a client would need to worry about.
>>
>> HTH,
>> Paul Davis
>>
>> On Sat, Sep 19, 2009 at 7:11 PM, Norman Barker <no...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > currently if adding documents to couch using the REST api it is
>> > possible to add a document such as
>> >
>> > {
>> >  "prop1": "1",
>> >  "prop1": "2"
>> > }
>> >
>> > which only displays partially in Futon (skips first property, "prop1":
>> > "1") due to the unique constraints of JSON object properties.
>> >
>> > This is actually a good feature, since I am generating invalid JSON
>> > from an event based API, is there a way to add a normalisation
>> > function to the write document operation  (using Erlang ideally) that
>> > would group these properties,  its gets difficult when the documents
>> > are nested but any thoughts appreciated.
>> >
>> > Many thanks,
>> >
>> > Norman
>> >
>>
>

Re: invalid json allowed into couch

[Nitin Borwankar <ni...@borwankar.com>]

Is this "feature" guaranteed to remain going forward?
My project has a lot of data that could use the repeated fields feature
(multiple authors of a bibliographic item, multiple keywords...)
but this is like pouring concrete - we can't uproot all of the data if a
decision is made later to be strictly JS compliant.
We are currently not using this capability, but it certainly looks
attractive and am curious about its longevity.

Damien or any other committers - any comments ?

Nitin


37% of all statistics are made up on the spot
-------------------------------------------------------------------------------------
Nitin Borwankar
nborwankar@gmail.com


On Sat, Sep 19, 2009 at 4:27 PM, Paul Davis <pa...@gmail.com>wrote:

> Technically, this isn't invalid JSON.
>
> Depending on how you interpret the spec, it says that properties only
> "MAY" be enforced as unique. Some people read the line about "being a
> subset of JavaScript" which does enforce uniqueness to override this
> sentence.
>
> Either way, the JSON serializer that CouchDB uses doesn't enforce
> uniqueness so it'll happily accept and repeat back objects with
> repeated property names. You can check with cURL that your doc
> actually kept both of those properties.
>
> As to having a 'fix' for this, the only thing that I could forsee
> happening is enforcing the constraint and just erroring out on the
> initial PUT request. Concatenating repeated fields would be something
> that a client would need to worry about.
>
> HTH,
> Paul Davis
>
> On Sat, Sep 19, 2009 at 7:11 PM, Norman Barker <no...@gmail.com>
> wrote:
> > Hi,
> >
> > currently if adding documents to couch using the REST api it is
> > possible to add a document such as
> >
> > {
> >  "prop1": "1",
> >  "prop1": "2"
> > }
> >
> > which only displays partially in Futon (skips first property, "prop1":
> > "1") due to the unique constraints of JSON object properties.
> >
> > This is actually a good feature, since I am generating invalid JSON
> > from an event based API, is there a way to add a normalisation
> > function to the write document operation  (using Erlang ideally) that
> > would group these properties,  its gets difficult when the documents
> > are nested but any thoughts appreciated.
> >
> > Many thanks,
> >
> > Norman
> >
>

Re: invalid json allowed into couch

[Paul Davis <pa...@gmail.com>]

On Sat, Sep 19, 2009 at 7:45 PM, Jesse Hallett <ha...@gmail.com> wrote:
> On Sat, Sep 19, 2009 at 4:27 PM, Paul Davis <pa...@gmail.com> wrote:
>> Technically, this isn't invalid JSON.
>>
>> Depending on how you interpret the spec, it says that properties only
>> "MAY" be enforced as unique. Some people read the line about "being a
>> subset of JavaScript" which does enforce uniqueness to override this
>> sentence.
>>
>> Either way, the JSON serializer that CouchDB uses doesn't enforce
>> uniqueness so it'll happily accept and repeat back objects with
>> repeated property names. You can check with cURL that your doc
>> actually kept both of those properties.
>>
>> As to having a 'fix' for this, the only thing that I could forsee
>> happening is enforcing the constraint and just erroring out on the
>> initial PUT request. Concatenating repeated fields would be something
>> that a client would need to worry about.
>>
>> HTH,
>> Paul Davis
>>
>> On Sat, Sep 19, 2009 at 7:11 PM, Norman Barker <no...@gmail.com> wrote:
>>> Hi,
>>>
>>> currently if adding documents to couch using the REST api it is
>>> possible to add a document such as
>>>
>>> {
>>>  "prop1": "1",
>>>  "prop1": "2"
>>> }
>>>
>>> which only displays partially in Futon (skips first property, "prop1":
>>> "1") due to the unique constraints of JSON object properties.
>>>
>>> This is actually a good feature, since I am generating invalid JSON
>>> from an event based API, is there a way to add a normalisation
>>> function to the write document operation  (using Erlang ideally) that
>>> would group these properties,  its gets difficult when the documents
>>> are nested but any thoughts appreciated.
>>>
>>> Many thanks,
>>>
>>> Norman
>>>
>>
>
> Could one use an _update function to combine duplicate keys?
>

Nope. Unless you write a view server that uses a JSON parser that
allows working with duplicated property names. Spidermonkey will just
hand you the same thing that Futon displays.

Re: invalid json allowed into couch

[Jesse Hallett <ha...@gmail.com>]

On Sat, Sep 19, 2009 at 4:27 PM, Paul Davis <pa...@gmail.com> wrote:
> Technically, this isn't invalid JSON.
>
> Depending on how you interpret the spec, it says that properties only
> "MAY" be enforced as unique. Some people read the line about "being a
> subset of JavaScript" which does enforce uniqueness to override this
> sentence.
>
> Either way, the JSON serializer that CouchDB uses doesn't enforce
> uniqueness so it'll happily accept and repeat back objects with
> repeated property names. You can check with cURL that your doc
> actually kept both of those properties.
>
> As to having a 'fix' for this, the only thing that I could forsee
> happening is enforcing the constraint and just erroring out on the
> initial PUT request. Concatenating repeated fields would be something
> that a client would need to worry about.
>
> HTH,
> Paul Davis
>
> On Sat, Sep 19, 2009 at 7:11 PM, Norman Barker <no...@gmail.com> wrote:
>> Hi,
>>
>> currently if adding documents to couch using the REST api it is
>> possible to add a document such as
>>
>> {
>>  "prop1": "1",
>>  "prop1": "2"
>> }
>>
>> which only displays partially in Futon (skips first property, "prop1":
>> "1") due to the unique constraints of JSON object properties.
>>
>> This is actually a good feature, since I am generating invalid JSON
>> from an event based API, is there a way to add a normalisation
>> function to the write document operation  (using Erlang ideally) that
>> would group these properties,  its gets difficult when the documents
>> are nested but any thoughts appreciated.
>>
>> Many thanks,
>>
>> Norman
>>
>

Could one use an _update function to combine duplicate keys?

Re: invalid json allowed into couch

[Paul Davis <pa...@gmail.com>]

Technically, this isn't invalid JSON.

Depending on how you interpret the spec, it says that properties only
"MAY" be enforced as unique. Some people read the line about "being a
subset of JavaScript" which does enforce uniqueness to override this
sentence.

Either way, the JSON serializer that CouchDB uses doesn't enforce
uniqueness so it'll happily accept and repeat back objects with
repeated property names. You can check with cURL that your doc
actually kept both of those properties.

As to having a 'fix' for this, the only thing that I could forsee
happening is enforcing the constraint and just erroring out on the
initial PUT request. Concatenating repeated fields would be something
that a client would need to worry about.

HTH,
Paul Davis

On Sat, Sep 19, 2009 at 7:11 PM, Norman Barker <no...@gmail.com> wrote:
> Hi,
>
> currently if adding documents to couch using the REST api it is
> possible to add a document such as
>
> {
>  "prop1": "1",
>  "prop1": "2"
> }
>
> which only displays partially in Futon (skips first property, "prop1":
> "1") due to the unique constraints of JSON object properties.
>
> This is actually a good feature, since I am generating invalid JSON
> from an event based API, is there a way to add a normalisation
> function to the write document operation  (using Erlang ideally) that
> would group these properties,  its gets difficult when the documents
> are nested but any thoughts appreciated.
>
> Many thanks,
>
> Norman
>