You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Jyrki Pulliainen (JIRA)" <ji...@apache.org> on 2010/04/19 11:10:50 UTC
[jira] Created: (COUCHDB-743) Sending attachments via new HTTP API
is slow if attachment size is bigger than 1024 bytes
Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
-----------------------------------------------------------------------------------------
Key: COUCHDB-743
URL: https://issues.apache.org/jira/browse/COUCHDB-743
Project: CouchDB
Issue Type: Bug
Components: Database Core, HTTP Interface
Affects Versions: 0.10.1, 0.10
Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
Reporter: Jyrki Pulliainen
#!/bin/sh
size=1025
When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
#!/bin/sh
size=1025
curl -X DELETE http://localhost:5984/test
curl -X PUT http://localhost:5984/test
docid=0
while true; do
rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
echo -n "start:"; date '+%H:%M:%S.%N'
head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
echo -n "end:"; date '+%H:%M:%S.%N'
docid=$(($docid+1))
done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-743) Sending attachments via new HTTP
API is slow if attachment size is bigger than 1024 bytes
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858441#action_12858441 ]
Filipe Manana commented on COUCHDB-743:
---------------------------------------
You sure it's a CouchDB problem?
I'm with the latest trunk version and for a file of size 2,2 Kbs, it takes 0.02 seconds the upload:
$ time --format=%e curl -X PUT http://localhost:5984/testedb/doc1/readme.txt?rev=1-17907786cd284d6585a640e97e0ff8b6 --data-binary @/home/fdmanana/git/couchdb/README -H 'Content-Type: text/plain'
{"ok":true,"id":"doc1","rev":"2-14094b637bf02c3052b9a9a04f0750ff"}
0.02
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-743) Sending attachments via new HTTP
API is slow if attachment size is bigger than 1024 bytes
Posted by "Jyrki Pulliainen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858443#action_12858443 ]
Jyrki Pulliainen commented on COUCHDB-743:
------------------------------------------
According to the discussion at IRC the problem exists only with 0.10.x releases, not with 0.11.0 or newer (trunk). However, the behaviour is reproducable in the 0.10-series with the attached script.
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-743) Sending attachments via new HTTP API
is slow if attachment size is bigger than 1024 bytes
Posted by "Jyrki Pulliainen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jyrki Pulliainen updated COUCHDB-743:
-------------------------------------
Description:
When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
#!/bin/sh
size=1025
curl -X DELETE http://localhost:5984/test
curl -X PUT http://localhost:5984/test
docid=0
while true; do
rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
echo -n "start:"; date '+%H:%M:%S.%N'
head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
echo -n "end:"; date '+%H:%M:%S.%N'
docid=$(($docid+1))
done
was:
#!/bin/sh
size=1025
When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
#!/bin/sh
size=1025
curl -X DELETE http://localhost:5984/test
curl -X PUT http://localhost:5984/test
docid=0
while true; do
rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
echo -n "start:"; date '+%H:%M:%S.%N'
head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
echo -n "end:"; date '+%H:%M:%S.%N'
docid=$(($docid+1))
done
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-743) Sending attachments via new HTTP
API is slow if attachment size is bigger than 1024 bytes
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858447#action_12858447 ]
Robert Newson commented on COUCHDB-743:
---------------------------------------
This could be curl's 1s timeout waiting for a 100 continue response that doesn't come. Can you rerun with -v to curl so we can see all the request and response headers?
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-743) Sending attachments via new HTTP
API is slow if attachment size is bigger than 1024 bytes
Posted by "Jyrki Pulliainen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858453#action_12858453 ]
Jyrki Pulliainen commented on COUCHDB-743:
------------------------------------------
We set the curl Expect header empty (using -H 'Expect:') and the problem seems to be gone. Thanks Robert Newson for the tip!
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-743) Sending attachments via new HTTP
API is slow if attachment size is bigger than 1024 bytes
Posted by "Jyrki Pulliainen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858450#action_12858450 ]
Jyrki Pulliainen commented on COUCHDB-743:
------------------------------------------
Seems like the 100 continue timeout. Below is the output of the curl
start:13:48:02.577094054
* About to connect() to 127.0.0.1 port 5984 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0)
> PUT /test/0/attachment?rev=1-967a00dff5e02add41819138abb3284d HTTP/1.1
> User-Agent: curl/7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:5984
> Accept: */*
> Content-Type: text/plain
> Content-Length: 1025
> Expect: 100-continue
>
* Done waiting for 100-continue
< HTTP/1.1 201 Created
< Server: CouchDB/0.10.0 (Erlang OTP/R13B)
< Location: http://127.0.0.1:5984/test/0/attachment
< Date: Mon, 19 Apr 2010 10:48:04 GMT
< Content-Type: text/plain;charset=utf-8
< Content-Length: 64
< Cache-Control: must-revalidate
<
{"ok":true,"id":"0","rev":"2-12b3cf68d8d3ef1279205787939e2e27"}
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
end:13:48:04.596758573
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-743) Sending attachments via new HTTP
API is slow if attachment size is bigger than 1024 bytes
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858452#action_12858452 ]
Robert Newson commented on COUCHDB-743:
---------------------------------------
For now, you can add -H "Expect: " to your curl command to suppress the expect header. CouchDB should immediately return a "HTTP/1.1 100 Continue\r\n" in this case, though.
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-743) Sending attachments via new HTTP API
is slow if attachment size is bigger than 1024 bytes
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-743:
--------------------------------------
Skill Level: Regular Contributors Level (Easy to Medium)
> Sending attachments via new HTTP API is slow if attachment size is bigger than 1024 bytes
> -----------------------------------------------------------------------------------------
>
> Key: COUCHDB-743
> URL: https://issues.apache.org/jira/browse/COUCHDB-743
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core, HTTP Interface
> Affects Versions: 0.10, 0.10.1
> Environment: Ubuntu Karmic (9.10) with CouchDB 0.10.0. According to IRC conversations, also reproduced with 0.10.1
> Reporter: Jyrki Pulliainen
>
> When using HTTP PUT to store attachments the request takes exactly one second to finish when the attachment size is 1025 bytes or more. This behaviour does not occur with inline attachments.
> Here's a bash script to reproduce the behaviour (requires curl and python >= 2.6)
> #!/bin/sh
> size=1025
> curl -X DELETE http://localhost:5984/test
> curl -X PUT http://localhost:5984/test
> docid=0
> while true; do
> rev=`curl -s -X PUT http://localhost:5984/test/$docid --data-binary '{}' | python -c "import json; import sys; print json.load(sys.stdin)['rev']"`
> echo -n "start:"; date '+%H:%M:%S.%N'
> head -c $size /dev/urandom | curl -s -X PUT -H 'Content-Type: text/plain' http://localhost:5984/test/$docid/attachment\?rev=$rev --data-binary @-
> echo -n "end:"; date '+%H:%M:%S.%N'
> docid=$(($docid+1))
> done
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.