You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Brian Candler <B....@pobox.com> on 2009/01/25 17:41:43 UTC

Issue with null MIME type for attachments

I am running couchdb SVN r736610 under Ubuntu Hardy (but with Erlang 12B3
packages from Intrepid)

After deploying sofa (git id 5222915fded972ddc55058c3fd97fa2bca366499) using
couchapp (gem 0.1.4), I notice that one of the attachments to _design/sofa
has a null content_type:

   "_attachments": {
       ...
       "to_couch.rb": {
           "stub": true,
           "content_type": null,
           "length": 1460
       },

And if I try to retrieve this using HTTP, I get an error:

$ curl http://127.0.0.1:5984/blogdb/_design/sofa/to_couch.rb
{"error":"error","reason":"badarg"}

Retrieving other attachments with non-null content types works fine.

Is it expected behaviour that a client app can upload an attachment in such
a way that it cannot be downloaded again? I think the server should either
fail the upload, or default the MIME type to application/octet-stream.

I see that couchapp is mapping MIME types via a Ruby hash:

    "content_type" => MIMES[path.split('.').last]

and since .rb is not known it is setting it to nil, which ends up as null in
the JSON.

The problem can be replicated easily without sofa/couchapp as follows:

$ curl -d '{"_attachments":{"foo.bin":{"content_type":null,"data":"aGVsbG8="}}}' http://127.0.0.1:5984/test_suite_db
{"ok":true,"id":"d166ec286f8fb05279e7770c225a610f","rev":"1376559224"}
$ curl http://127.0.0.1:5984/test_suite_db/d166ec286f8fb05279e7770c225a610f
{"_id":"d166ec286f8fb05279e7770c225a610f","_rev":"1376559224","_attachments":{"foo.bin":{"stub":true,"content_type":null,"length":5}}}
$ curl http://127.0.0.1:5984/test_suite_db/d166ec286f8fb05279e7770c225a610f/foo.bin
{"error":"error","reason":"badarg"}

Regards,

Brian.

Re: Issue with null MIME type for attachments

Posted by Michael McDaniel <co...@autosys.us>.
On Sun, Jan 25, 2009 at 10:10:10AM -0800, Chris Anderson wrote:
> On Sun, Jan 25, 2009 at 8:41 AM, Brian Candler <B....@pobox.com> wrote:
> 
> > Is it expected behaviour that a client app can upload an attachment in such
> > a way that it cannot be downloaded again? I think the server should either
> > fail the upload, or default the MIME type to application/octet-stream.
> 
> This is definitely a bug in Ruby couchapp, but you raise an
> interesting question.
> 
> Anyone have a strong opinion on which way to go (default vs error)?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 1) If a proper upload is defined that app must provide a MIME type,
    and none is provided by the app, return an error.  

 2) Define an upload as "provide your MIME type, or you'll get X".
    And it wouldn't hurt to send back info to the app that
    "MIME type was set to X".


 I suggest '1)'

~Michael


> 
> Chris
> 
> -- 
> Chris Anderson
> http://jchris.mfdz.com

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us


Re: Issue with null MIME type for attachments

Posted by Noah Slater <ns...@apache.org>.
On Mon, Jan 26, 2009 at 07:04:31AM +1030, Antony Blakey wrote:
>
> On 26/01/2009, at 6:55 AM, Noah Slater wrote:
>
>> On Mon, Jan 26, 2009 at 06:49:53AM +1030, Antony Blakey wrote:
>>> Treat a supplied null as an error.
>>>
>>> If mimetype isn't supplied, use a default based on the attachment
>>> name.
>>
>> Default based on attachment name sounds misleadingly simple
>>
>> I agree that this should be the way forward but we will need to:
>>
>>  * ship a custom mime.types configuration file under /etc
>>  * know how to read the system mime.types file usually at /etc/
>> mime.types
>>  * know how to pick up any other user supplied mime.types file
>>  * add configuration options for this in the ini files
>
> And deal with Windows.

Windows isn't supported. :)

-- 
Noah Slater, http://tumbolia.org/nslater

Re: Issue with null MIME type for attachments

Posted by Antony Blakey <an...@gmail.com>.
On 26/01/2009, at 6:55 AM, Noah Slater wrote:

> On Mon, Jan 26, 2009 at 06:49:53AM +1030, Antony Blakey wrote:
>> Treat a supplied null as an error.
>>
>> If mimetype isn't supplied, use a default based on the attachment  
>> name.
>
> Default based on attachment name sounds misleadingly simple
>
> I agree that this should be the way forward but we will need to:
>
>  * ship a custom mime.types configuration file under /etc
>  * know how to read the system mime.types file usually at /etc/ 
> mime.types
>  * know how to pick up any other user supplied mime.types file
>  * add configuration options for this in the ini files

And deal with Windows.

Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

A reasonable man adapts himself to suit his environment. An  
unreasonable man persists in attempting to adapt his environment to  
suit himself. Therefore, all progress depends on the unreasonable man.
   -- George Bernard Shaw



Re: Issue with null MIME type for attachments

Posted by Brian Candler <B....@pobox.com>.
On Sun, Jan 25, 2009 at 09:06:22PM +0000, Brian Candler wrote:
> For consistency, whatever logic chosen should also apply to uploads of
> standalone attachments without any Content-Type header. As far as I can tell
> using telnet, these give a badarg response at the moment.

And the other case I hadn't considered was inline attachment with no
content_type specified at all:

$ curl -d '{"_attachments":{"foo.txt":{"data":"aGVsbG8="}}}' http://127.0.0.1:5984/test_suite_db; echo
{"ok":true,"id":"463a4a56aed36c3de44cae6f35df2507","rev":"2681060313"}
$ curl http://127.0.0.1:5984/test_suite_db/463a4a56aed36c3de44cae6f35df2507; echo
{"_id":"463a4a56aed36c3de44cae6f35df2507","_rev":"2681060313","_attachments":{"foo.txt":{"stub":true,"content_type":"application/octet-stream","length":5}}}

Ah, so *now* you get application/octet-stream :-)

Summarising current behaviour:

1. uploading attachment with no Content-Type: header: fails
2. inline attachment with no content_type: gives application/octet-stream
3. inline attachment with bad (non-string) content_type: stores the bad value,
   and this gives badarg when you later try to download it

I think (1) should behave as (2), and (3) should behave as (1).

However I wouldn't object if "content_type":null was treated more gracefully
as if content_type were absent entirely.

Regards,

Brian.

Re: Issue with null MIME type for attachments

Posted by Noah Slater <ns...@apache.org>.
On Sun, Jan 25, 2009 at 09:45:27PM +0000, Brian Candler wrote:
> On Sun, Jan 25, 2009 at 09:38:45PM +0000, Noah Slater wrote:
> > On Sun, Jan 25, 2009 at 09:06:22PM +0000, Brian Candler wrote:
> > > For now though, couchapp could use the ruby MIME::Types gem in place of its
> > > own hard-coded table.
> > >
> > > irb(main):008:0> MIME::Types.type_for('citydesk.xml').first.to_s
> > > => "application/xml"
> >
> > If Ruby is anything like Python, it pulls those values from /etc/mime.types
>
> Not the Ruby one. It has them all hardcoded direct from the RFCs, plus some
> metadata which /etc/mime.types doesn't include (for example, it notes which
> MIME types refer to digital signatures)
>
> I guess if the server did this, and did it properly, it would free up all
> the clients from having to worry about it.

Yep, this is what we want to do.

Apache httpd provides a way to configure this using regular configuration files,
but as this wouldn't comfortably fit into an ini format, we would do well to
just use a standard mime.types file and let users specify multiple ones. Merging
these files is easy as pie.

-- 
Noah Slater, http://tumbolia.org/nslater

Re: Issue with null MIME type for attachments

Posted by Brian Candler <B....@pobox.com>.
On Sun, Jan 25, 2009 at 09:38:45PM +0000, Noah Slater wrote:
> On Sun, Jan 25, 2009 at 09:06:22PM +0000, Brian Candler wrote:
> > For now though, couchapp could use the ruby MIME::Types gem in place of its
> > own hard-coded table.
> >
> > irb(main):008:0> MIME::Types.type_for('citydesk.xml').first.to_s
> > => "application/xml"
> 
> If Ruby is anything like Python, it pulls those values from /etc/mime.types

Not the Ruby one. It has them all hardcoded direct from the RFCs, plus some
metadata which /etc/mime.types doesn't include (for example, it notes which
MIME types refer to digital signatures)

I guess if the server did this, and did it properly, it would free up all
the clients from having to worry about it.

Re: Issue with null MIME type for attachments

Posted by Noah Slater <ns...@apache.org>.
On Sun, Jan 25, 2009 at 09:06:22PM +0000, Brian Candler wrote:
> For now though, couchapp could use the ruby MIME::Types gem in place of its
> own hard-coded table.
>
> irb(main):008:0> MIME::Types.type_for('citydesk.xml').first.to_s
> => "application/xml"

If Ruby is anything like Python, it pulls those values from /etc/mime.types

... at least, from memory it does. Been a while.

-- 
Noah Slater, http://tumbolia.org/nslater

Re: Issue with null MIME type for attachments

Posted by Brian Candler <B....@pobox.com>.
On Sun, Jan 25, 2009 at 08:25:48PM +0000, Noah Slater wrote:
> On Mon, Jan 26, 2009 at 06:49:53AM +1030, Antony Blakey wrote:
> > Treat a supplied null as an error.
> >
> > If mimetype isn't supplied, use a default based on the attachment name.
> 
> Default based on attachment name sounds misleadingly simple
> 
> I agree that this should be the way forward but we will need to:
> 
>   * ship a custom mime.types configuration file under /etc
>   * know how to read the system mime.types file usually at /etc/mime.types
>   * know how to pick up any other user supplied mime.types file
>   * add configuration options for this in the ini files
> 
> Check your local mime.types file for more hints. The format is pretty simple.

For consistency, whatever logic chosen should also apply to uploads of
standalone attachments without any Content-Type header. As far as I can tell
using telnet, these give a badarg response at the moment.

RFC 2616 allows you to guess based on extension, but also allows you to
default to application/octet-stream. (7.2.1)

   Any HTTP/1.1 message containing an entity-body SHOULD include a
   Content-Type header field defining the media type of that body. If
   and only if the media type is not given by a Content-Type field, the
   recipient MAY attempt to guess the media type via inspection of its
   content and/or the name extension(s) of the URI used to identify the
   resource. If the media type remains unknown, the recipient SHOULD
   treat it as type "application/octet-stream".

For now though, couchapp could use the ruby MIME::Types gem in place of its
own hard-coded table.

irb(main):008:0> MIME::Types.type_for('citydesk.xml').first.to_s
=> "application/xml"

Regards,

Brian.

Re: Issue with null MIME type for attachments

Posted by Noah Slater <ns...@apache.org>.
On Mon, Jan 26, 2009 at 06:49:53AM +1030, Antony Blakey wrote:
> Treat a supplied null as an error.
>
> If mimetype isn't supplied, use a default based on the attachment name.

Default based on attachment name sounds misleadingly simple

I agree that this should be the way forward but we will need to:

  * ship a custom mime.types configuration file under /etc
  * know how to read the system mime.types file usually at /etc/mime.types
  * know how to pick up any other user supplied mime.types file
  * add configuration options for this in the ini files

Check your local mime.types file for more hints. The format is pretty simple.

-- 
Noah Slater, http://tumbolia.org/nslater

Re: Issue with null MIME type for attachments

Posted by Brian Candler <B....@pobox.com>.
On Mon, Jan 26, 2009 at 07:16:18AM +1030, Antony Blakey wrote:
>> Or indeed anything which is not a string. At the moment, arbitrary  
>> JSON
>> values are accepted, such as
>>
>>    "content_type":[{"foo":"bar"}]
>
> A bug, although I'm curious what mimetype is thereafter supplied by a  
> GET on the attachment.

The same as with "content_type":null - the entire GET fails with a "badarg".

$ curl http://127.0.0.1:5984/test_suite_db/66b7393fab5a2c1b68a6a8b007b8150d
{"_id":"66b7393fab5a2c1b68a6a8b007b8150d","_rev":"3037347223","_attachments":{"badattach.html":{"stub":true,"content_type":"image/jpeg","length":8},"foo.bin":{"stub":true,"content_type":[{"foo":"bar"}],"length":5}}}
$ curl http://127.0.0.1:5984/test_suite_db/66b7393fab5a2c1b68a6a8b007b8150d/foo.bin
{"error":"error","reason":"badarg"}

Re: Issue with null MIME type for attachments

Posted by Antony Blakey <an...@gmail.com>.
On 26/01/2009, at 7:11 AM, Brian Candler wrote:

> On Mon, Jan 26, 2009 at 06:49:53AM +1030, Antony Blakey wrote:
>> Treat a supplied null as an error.
>
> Or indeed anything which is not a string. At the moment, arbitrary  
> JSON
> values are accepted, such as
>
>    "content_type":[{"foo":"bar"}]

A bug, although I'm curious what mimetype is thereafter supplied by a  
GET on the attachment.

Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

Isn't it enough to see that a garden is beautiful without having to  
believe that there are fairies at the bottom of it too?
   -- Douglas Adams


Re: Issue with null MIME type for attachments

Posted by Brian Candler <B....@pobox.com>.
On Mon, Jan 26, 2009 at 06:49:53AM +1030, Antony Blakey wrote:
> Treat a supplied null as an error.

Or indeed anything which is not a string. At the moment, arbitrary JSON
values are accepted, such as

    "content_type":[{"foo":"bar"}]

Re: Issue with null MIME type for attachments

Posted by Antony Blakey <an...@gmail.com>.
On 26/01/2009, at 4:40 AM, Chris Anderson wrote:

> On Sun, Jan 25, 2009 at 8:41 AM, Brian Candler <B....@pobox.com>  
> wrote:
>
>> Is it expected behaviour that a client app can upload an attachment  
>> in such
>> a way that it cannot be downloaded again? I think the server should  
>> either
>> fail the upload, or default the MIME type to application/octet- 
>> stream.
>
> This is definitely a bug in Ruby couchapp, but you raise an
> interesting question.
>
> Anyone have a strong opinion on which way to go (default vs error)?

Treat a supplied null as an error.

If mimetype isn't supplied, use a default based on the attachment name.

Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

It is no measure of health to be well adjusted to a profoundly sick  
society.
   -- Jiddu Krishnamurti



Re: Issue with null MIME type for attachments

Posted by Chris Anderson <jc...@apache.org>.
On Sun, Jan 25, 2009 at 8:41 AM, Brian Candler <B....@pobox.com> wrote:

> Is it expected behaviour that a client app can upload an attachment in such
> a way that it cannot be downloaded again? I think the server should either
> fail the upload, or default the MIME type to application/octet-stream.

This is definitely a bug in Ruby couchapp, but you raise an
interesting question.

Anyone have a strong opinion on which way to go (default vs error)?

Chris

-- 
Chris Anderson
http://jchris.mfdz.com