You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "avarf (via GitHub)" <gi...@apache.org> on 2023/11/16 10:56:21 UTC

[I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

avarf opened a new issue, #4851:
URL: https://github.com/apache/couchdb/issues/4851

   
   ## Description
   I am using the same function in different flows to upload/add an attachment to a document and it was always working fine until recently in one specific flow. 
   
   We have a json file that is the status of another microservice that we want to store. If we get the status while the process is running and try to store it couchDB returns 201 (and I can see 201 in couchDB logs too) but the attachment is missing and I can't neither fetch it nor see it in the GUI. BUT if we let that third-party process end by itself and then fetch the status I am able to store it as an attachment and later fetch it.
   
   Since I can't see any strange thing in the status that I want to store as an attachment when I print them in logs I have no idea why this happens. Can a hidden character/binary value trigger this behavior in couchDB?
   
   I want to emphasize that this code and setup works totally fine IF that third-party process ends or if we want to add some other attachment.
   
   ## Steps to Reproduce
   Since I can't provide you access to our product I only provide the logs and the code that I use for adding the attachment.
   
   Code:
   ```go
   func PutAttachment(url string, attachmentName string, attachment []byte) error {
   	log.Debug("url: " + url)
   	log.Debug("attachmentName: " + attachmentName)
   
   	ETag, err := HeadDoc(url)
   	if err != nil {
   		log.Error(err.Error())
   		return err
   	}
   	log.Info("ETag: " + ETag)
   
   	req, err := http.NewRequest("PUT", url+"/"+attachmentName, bytes.NewBuffer(attachment))
   	if err != nil {
   		returnErr := errors.New("error http POST request: " + err.Error())
   		log.Error(returnErr)
   		return returnErr
   	}
   
   	req.Header.Set("Content-Type", "application/json")
   	req.Header.Add("Authorization", "Basic "+basicAuth(COUCHDB_USERNAME, COUCHDB_PASSWORD))
   	if ETag != "" {
   		req.Header.Add("If-Match", ETag)
   	}
   	log.Trace(req.Header)
   
   	response, err := http.DefaultClient.Do(req)
   	if err != nil {
   		returnErr := errors.New("Error getting response from DB: " + err.Error())
   		log.Error(returnErr)
   		return returnErr
   	}
   
   	defer response.Body.Close()
   
   	if response.StatusCode > 202 {
   		returnErr := errors.New("Error PUTing attachment to DB. Status code: " + response.Status)
   		log.Error(returnErr)
   		return returnErr
   	}
   	log.Trace(response)
   	log.Trace(response.StatusCode)
   
   	// return gjson.ParseBytes(bodyBytes).Value().(map[string]interface{}), nil
   	return nil
   
   }
   ```
   
   My component logs:
   ```log
   time="2023-11-16T10:46:21Z" level=debug msg="url: http://couchdb:5984/experiments/sys_p1_e1" func=src/db.PutAttachment file="/src/db/db.go:452"
   time="2023-11-16T10:46:21Z" level=debug msg="attachmentName: simulation_result" func=src/db.PutAttachment file="/src/db/db.go:453"
   time="2023-11-16T10:46:21Z" level=trace msg="url: http://couchdb:5984/experiments/sys_p1_e1" func=src/db.HeadDoc file="/src/db/db.go:232"
   time="2023-11-16T10:46:21Z" level=debug msg="Head response.StatusCode: 200 OK" func=src/db.HeadDoc file="/src/db/db.go:246"
   time="2023-11-16T10:46:21Z" level=debug msg="ETag: <<\"37-cfa844a91434c910f936b5edd80c0fc9\">>" func=src/db.HeadDoc file="/src/db/db.go:250"
   time="2023-11-16T10:46:21Z" level=info msg="ETag: \"37-cfa844a91434c910f936b5edd80c0fc9\"" func=src/db.PutAttachment file="/src/db/db.go:462"
   time="2023-11-16T10:46:21Z" level=trace msg="map[Authorization:[Basic YWRtaW46YWRtaW4=] Content-Type:[application/json] If-Match:[\"37-cfa844a91434c910f936b5edd80c0fc9\"]]" func=src/db.PutAttachment file="/src/db/db.go:476"
   time="2023-11-16T10:46:21Z" level=trace msg="&{201 Created 201 HTTP/1.1 1 1 map[Cache-Control:[must-revalidate] Content-Length:[73] Content-Type:[application/json] Date:[Thu, 16 Nov 2023 10:46:20 GMT] Location:[http://couchdb:5984/experiments/sys_p1_e1/simulation_result] Server:[CouchDB/3.2.2 (Erlang OTP/23)] X-Couch-Request-Id:[70c070dc24] X-Couchdb-Body-Time:[0]] 0xc0003d27c0 73 [] false false map[] 0xc0000ab300 <nil>}" func=src/db.PutAttachment file="/src/db/db.go:492"
   time="2023-11-16T10:46:21Z" level=trace msg=201 func=src/db.PutAttachment file="/src/db/db.go:493"
   time="2023-11-16T10:46:21Z" level=debug msg="simulation_result attached to the expDoc" func=src/experiment.StopExperiment file="/src/experiment/experiment.go:592"
   ```
   
   couchDB logs:
   ```log
   [notice] 2023-11-16T10:46:21.336904Z nonode@nohost <0.17885.98> 21c0838990 couchdb:5984 172.24.0.9 admin GET /experiments/sys_p1_e1 200 ok 1
   [notice] 2023-11-16T10:46:21.590848Z nonode@nohost <0.17885.98> bcd75ce1fc couchdb:5984 172.24.0.9 admin HEAD /experiments/sys_p1_e1 200 ok 4
   [notice] 2023-11-16T10:46:21.620233Z nonode@nohost <0.17885.98> 70c070dc24 couchdb:5984 172.24.0.9 admin PUT /experiments/sys_p1_e1/simulation_result 201 ok 28
   [notice] 2023-11-16T10:46:21.628153Z nonode@nohost <0.17921.98> 89d320dece couchdb:5984 172.24.0.9 admin HEAD /experiments/sys_p1_e1 200 ok 3
   [notice] 2023-11-16T10:46:21.644016Z nonode@nohost <0.17921.98> 725a766ed8 couchdb:5984 172.24.0.9 admin PUT /experiments/sys_p1_e1 201 ok 15
   ```
   
   ## Expected Behaviour
   I expect if there is something wrong with the attachment I receive an error and not a 201 success.
   
   ## Your Environment
   
   [TIP]:  # ( Include as many relevant details about your environment as possible. )
   [TIP]:  # ( You can paste the output of curl http://YOUR-COUCHDB:5984/ here. )
   
   * CouchDB version used:
   ```yaml
     couchdb:
       image: couchdb:3.2.2
       restart: always
       ports:
         - '5984:5984'
       environment:
         - COUCHDB_USER=USERNAME
         - COUCHDB_PASSWORD=PASSWORD
       volumes:
         - ./dbdata:/opt/couchdb/data
         - ./local.ini:/opt/couchdb/etc/local.d/local.ini
       networks:
         - local-enorch
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #4851:
URL: https://github.com/apache/couchdb/issues/4851#issuecomment-1814620175

   attachments are "attached" to the documents, indeed. you can delete documents by removing the _attachments object (or items within it).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #4851:
URL: https://github.com/apache/couchdb/issues/4851#issuecomment-1814618423

   aaah!
   
   Are you wiping out the _attachments meta object when updating the document?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #4851:
URL: https://github.com/apache/couchdb/issues/4851#issuecomment-1814765361

   all the `_` prefixed attributes have a meaning to couchdb (the rest are user data), so _id, _rev, _deleted and _attachments all matter.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "avarf (via GitHub)" <gi...@apache.org>.
avarf commented on issue #4851:
URL: https://github.com/apache/couchdb/issues/4851#issuecomment-1814614474

   The issue is I have a PUT call to update the document later down the flow and it is updating the document with up-to-date information BUT it is not aware of `_attachments` and overwrites it and because of that CouchDB doesn't show the attachment.
   
   I was thinking that attachments are some how independent from the content of the document but it seems that it is not true.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #4851:
URL: https://github.com/apache/couchdb/issues/4851#issuecomment-1814266006

   Can you fetch the document with `?conflicts=true` query parameter? perhaps you have introduced conflicts and the attachment is stored with a "losing" revision. 
   
   We've not seen a 201 response for an attachment PUT that didn't mean the attachment was correctly stored. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "avarf (via GitHub)" <gi...@apache.org>.
avarf commented on issue #4851:
URL: https://github.com/apache/couchdb/issues/4851#issuecomment-1814686475

   Yes, I was deleting the _attachments meta while updating the document.
   
   What other meta do I need to be careful about? :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Using the same code to PUT an attachment in some specific cases the attachment is missing [couchdb]

Posted by "avarf (via GitHub)" <gi...@apache.org>.
avarf closed issue #4851: Using the same code to PUT an attachment in some specific cases the attachment is missing
URL: https://github.com/apache/couchdb/issues/4851


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org