You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2021/11/02 08:31:00 UTC
[GitHub] [couchdb-erlfdb] wfnuser opened a new issue #30: `Close` api is not supported?
wfnuser opened a new issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30
Hi, there. I looked through the erlfdb source code.
I'm surprised that `close connection` api is not supported? If it is true, I'm wondering why do you design like that?
How can I actively disconnect the connection to FDB?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-959136977
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-959136977
Hi, CometDB looks neat!
I believe the FDB C client will take care to automatically reconnect. Language bindings like ours don't need to worry about the management of anything at the TCP layer. What you _do_ need to worry about is that a networking blip could cause a transaction to return with `commit_unknown_result` instead of explicitly committing or aborting. In that case you want to have some logic to retry when networking is restored and see if the transaction succeeded, and retry if it did not. [Here's](https://github.com/apache/couchdb/blob/701d9663d349cce9c175b6ba75ea074889c35633/src/fabric/src/fabric2_fdb.erl#L185-L190) where we do that in CouchDB.
Your second question is a good one. We may actually have a small leak in the way we are managing DB handles when configuration settings are changed online. In CouchDB we are acquiring a DB handle [here](https://github.com/apache/couchdb/blob/082f8078411aab7d71cc86afca0fe2eff3104b01/src/fabric/src/fabric2_server.erl#L135), which we drop into the application environment so any process can use it directly. But then if a CouchDB admin changes certain config properties we will re-create the handle [here](https://github.com/apache/couchdb/blob/082f8078411aab7d71cc86afca0fe2eff3104b01/src/fabric/src/fabric2_server.erl#L162-L168), and we don't clean up the old one. Of course the same thing would happen if the `fabric2_server` dies. In practice it hasn't been a concern, this process is stable and the DB handles are very small resources, but I'll ask around and see if I'm missing something.
One other comment - it looks like you're planning to issue a call to a `comet` gen_server every time you want do an operation on a queue. You may not need to do that. You could avoid the message passing overhead by publishing the DB handle to an ets table, the application environment, or a [persistent_term](https://www.erlang.org/doc/man/persistent_term.html).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-961598998
Thanks again for everyone. π
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-961598998
Thanks again for everyone. π
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-957547020
Good question! Yes, the only place where we stop the network thread is when the erlfdb code is purged from the VM.
The FoundationDB C client code has this property that there is at most one network thread for communication between a client and a given cluster for the life of the client process. It is possible to stop that thread with `fdb_stop_network()`, but once you do that you cannot start the thread again. More details on that design here:
https://apple.github.io/foundationdb/api-general.html#client-network-thread
So we are really just aligning to the standard FoundationDB client/server pattern here: a client maintains a single, long-lived connection to the server, and all transactions are multiplexed onto that connection. Each transaction can be committed / aborted independently.
If you _really_ needed to close the connection but keep the Erlang VM running you might be able to do it by purging erlfdb, and then loading it back in again later if you needed to reconnect, but in practice this isn't something that we've had any need to do (or have done any testing around).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] janl commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
janl commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-961050290
@wfnuser welcome aboard ;) β We are on [IRC](https://web.libera.chat/#couchdb) and [Slack](https://join.slack.com/t/couchdb/shared_invite/zt-fa9zim0j-H04m4o_KcLdWeOxEAcwM8g) β thatβs probably best to get started :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-958795683
Thanks! I'm very new to erlang, so I have two question more:
1. what will happen if the connection is closed for the reason such as `the broken network`... we should reconnect or do something else?
2. I'm using your code to build a data model on FDB. https://github.com/wfnuser/CometDB/blob/10d30739828361792bee827e1e43974b1c697074/src/comet.erl#L50
I'm wondering if the process is killed for some unpredictable reason. We should purge the connection here? How?
If you can show me where the code of couchdb handle this case will be very helpful.
Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-958795683
Thanks! I'm very new to erlang, so I have two question more:
1. what will happen if the connection is closed for the reason such as `the broken network`... we should reconnect or do something else?
2. I'm using your code to build a data model on FDB. https://github.com/wfnuser/CometDB/blob/10d30739828361792bee827e1e43974b1c697074/src/comet.erl#L50
I'm wondering if the process is killed for some unpredictable reason. We should purge the connection here? How?
If you can show me where the code of couchdb handle this case will be very helpful.
Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-957547020
Good question! Yes, the only place where we stop the network thread is when the erlfdb code is purged from the VM.
The FoundationDB C client code has this property that there is at most one network thread for communication between a client and a given cluster for the life of the client process. It is possible to stop that thread with `fdb_stop_network()`, but once you do that you cannot start the thread again. More details on that design here:
https://apple.github.io/foundationdb/api-general.html#client-network-thread
So we are really just aligning to the standard FoundationDB client/server pattern here: a client maintains a single, long-lived connection to the server, and all transactions are multiplexed onto that connection. Each transaction can be committed / aborted independently.
If you _really_ needed to close the connection but keep the Erlang VM running you might be able to do it by purging erlfdb, and then loading it back in again later if you needed to reconnect, but in practice this isn't something that we've had any need to do (or have done any testing around).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-957547020
Good question! Yes, the only place where we stop the network thread is when the erlfdb code is purged from the VM.
The FoundationDB C client code has this property that there is at most one network thread for communication between a client and a given cluster for the life of the client process. It is possible to stop that thread with `fdb_stop_network()`, but once you do that you cannot start the thread again. More details on that design here:
https://apple.github.io/foundationdb/api-general.html#client-network-thread
So we are really just aligning to the standard FoundationDB client/server pattern here: a client maintains a single, long-lived connection to the server, and all transactions are multiplexed onto that connection. Each transaction can be committed / aborted independently.
If you _really_ needed to close the connection but keep the Erlang VM running you might be able to do it by purging erlfdb, and then loading it back in again later if you needed to reconnect, but in practice this isn't something that we've had any need to do (or have done any testing around).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser closed issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser closed issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-960474455
Wow! Thanks a lot, especially for the advices about `comet`.
You're so nice. I would like to help to contribute CouchDB, is there any issues you recommend me to get start from.
Maybe I can get in touch more instantly with you guys over slack channels or some other ways.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-958795683
Thanks! I'm very new to erlang, so I have two question more:
1. what will happen if the connection is closed for the reason such as `the broken network`... we should reconnect or do something else?
2. I'm using your code to build a data model on FDB. https://github.com/wfnuser/CometDB/blob/10d30739828361792bee827e1e43974b1c697074/src/comet.erl#L50
I'm wondering if the process is killed for some unpredictable reason. We should purge the connection here? How?
If you can show me where the code of couchdb handle this case will be very helpful.
Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser closed issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser closed issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-959154731
Ah, right after I posted this I remembered how we clean up those DB handles! We basically just rely on the built-in Erlang reference counting. When the reference count for a DB handle drops to zero, Erlang will call the destructor that we defined [here](https://github.com/apache/couchdb-erlfdb/blob/818777328c5fbebb331fd9c69b1095b172ed8397/c_src/resources.c#L80-L88), which in turn calls `fdb_database_destroy`. So no, you don't need to do any explicit handle cleanup π
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] kocolosk commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
kocolosk commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-959136977
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-958795683
Thanks! I'm very new to erlang, so I have two question more:
1. what will happen if the connection is closed for the reason such as `the broken network`... we should reconnect or do something else?
2. I'm using your code to build a data model on FDB. https://github.com/wfnuser/CometDB/blob/10d30739828361792bee827e1e43974b1c697074/src/comet.erl#L50
I'm wondering if the process is killed for some unpredictable reason. We should purge the connection here? How?
If you can show me where the code of couchdb handle this case will be very helpful.
Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-erlfdb] wfnuser commented on issue #30: `Close` api is not supported?
Posted by GitBox <gi...@apache.org>.
wfnuser commented on issue #30:
URL: https://github.com/apache/couchdb-erlfdb/issues/30#issuecomment-961598998
Thanks again for everyone. π
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org