[GitHub] [couchdb-erlfdb] wfnuser opened a new issue #45: How to store Types like Atom using couchdb-erlfdb.

[GitBox <gi...@apache.org>]

wfnuser opened a new issue #45:
URL: https://github.com/apache/couchdb-erlfdb/issues/45


   Hi, I'm coming again. 
   The problem is that I would like to store tuples into FoundationDB.
   But it seems it won't work fine if I want to store tuples with atom() in it to FoundaitonDB.
   `#{context => {invalid_tuple_term,message},exception => error,stacktrace => [{erlfdb_tuple,encode,2,[{file,"er│me has already been fetched lfdb_tuple.erl"},{line,258}]}`
   
   Furthermore, If I want to store types other than binary(), the only way seems to be call `erlfdb_tuple:pack()` ? 
   Any advice if I want store arbitrary type in foundationdb? That will help a lot.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [couchdb-erlfdb] kocolosk commented on issue #45: How to store Types like Atom using couchdb-erlfdb.

[GitBox <gi...@apache.org>]

kocolosk commented on issue #45:
URL: https://github.com/apache/couchdb-erlfdb/issues/45#issuecomment-968964212


   That's correct, `erlfdb_tuple` is this library's module for encoding data types into binaries following FoundationDB's Tuple specification. It's not required for a FoundationDB client to follow that spec, but it's a good one, and ensures that different FDB clients can interoperate on the same set of data. We could probably do with some more worked examples of creating the Erlang tuples that `erlfdb_tuple` expects as input.
   
   I see at least a couple of options for handling a broader spectrum of data types:
   
   ### User-Specific Type Codes
   
   The FDB Tuple spec reserves a set of type codes for user-specific data types that do not map directly to one of the supported data types already defined in the spec. We could choose to allocate type codes within that space to support other Erlang data types. For example, we could declare that `0x40` would prefix a binary that was encoded in the [Erlang External Term Format](https://www.erlang.org/doc/apps/erts/erl_ext_dist.html), i.e. the output of the `term_to_binary` function, which would allow us to serialize arbitrary Erlang terms in FoundationDB and automatically decode them on the way out. I would caution against using that capability for the Keys of records in FoundationDB, as I don't really think the External Term Format has been designed with stable sorting in mind, but it could prove useful for intermixing arbitrary Erlang terms and other data types that are natively supported in FoundationDB as part of the values. This would require a small patch to `erlfdb`
   
   ### Ignore Tuple Layer for Values
   
   Alternatively, your application could decide that _every_ Value your app stores with FoundationDB is encoded with `term_to_binary` and ignore the Tuple layer altogether. You'd lose the ability for other FDB clients to parse the Values, but you'd have an easier native Erlang experience, e.g.
   
   ```erlang
   Eshell V12.1.4  (abort with ^G)
   1> DB = erlfdb:open().
   {erlfdb_database,#Ref<0.289576361.3161587713.93177>}
   2> Key = erlfdb_tuple:pack({<<"foo">>, 123}).
   <<1,102,111,111,0,21,123>>
   3> erlfdb:set(DB, Key, term_to_binary(hello)).
   ok
   4> erlfdb:get(DB, Key).
   <<131,100,0,5,104,101,108,108,111>>
   5> binary_to_term(erlfdb:get(DB, Key)).
   hello
   ```
   
   You'll see I still used the Tuple encoding for Keys, as it's carefully designed for proper sorting that's essential to most data modeling techniques in FoundationDB.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org