You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Michael Ivanov <iv...@logit-ag.de> on 2015/02/13 22:22:09 UTC
pn_data_format
Hallo,
I am trying to use proton routine for data formatting.
pn_data_format checks suire of provided buffer and in case it is not
enough just refuses to copy anything at all.
Is it possible to find out how much does it need beforehand?
Ideally would be if one had the possibility to get the pointer to this
internal string (data->str) which already holds the string representation
of data. Does this option exist?
Best regards,
--
\ / | |
(OvO) | Mikhail Iwanow |
(^^^) | Voice: +7 (911) 223-1300 |
\^/ | E-mail: ivans@logit-ag.de |
^ ^ | |
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: pn_data_format
Posted by Rafael Schloming <rh...@alum.mit.edu>.
As Fraser says, you can't access the internal string, but you do have the
option to create your own externally:
pn_data_t *data = ...;
pn_string_t *str = pn_string(NULL);
pn_inspect(data, str);
const char *result = pn_string_get(str);
... // use result
pn_free(str); // you can also use pn_incref(str)/pn_decref(str) to
share a single pn_string_t if you want.
// pn_free() just asserts that pn_refcount(str) is
one and then pn_decref()s it
This won't tell you in advance how much space you need, but it will
dynamically grow the buffer inside the pn_string_t for you.
--Rafael
On Sat, Feb 14, 2015 at 2:40 AM, Fraser Adams <fraser.adams@blueyonder.co.uk
> wrote:
> Hello Mikhail,
> I'm not an expert on pn_data_format and TBH I've never used it in pure C
> code, only to write other language bindings. I don't know if it's possible
> to find out beforehand. My guess is probably not - the python binding
> (which is fairly canonical) looks like this:
>
>
> def format(self):
> sz = 16
> while True:
> err, result = pn_data_format(self._data, sz)
> if err == PN_OVERFLOW:
> sz *= 2
> continue
> else:
> self._check(err)
> return result
>
>
> And I borrowed that pattern when I did the JavaScript binding, which looks
> like this:
>
>
> /**
> * Format the encoded AMQP Data into a string representation and return it.
> * @method format
> * @memberof! proton.Data#
> * @returns {string} a formatted string representation of the encoded Data.
> */
> _Data_['format'] = function() {
> var sp = Runtime.stackSave();
> var sizeptr = allocate(4, 'i32', ALLOC_STACK);
>
> var size = 1024; // Pass by reference variable - need to use setValue
> to initialise it.
> while (true) {
> setValue(sizeptr, size, 'i32'); // Set pass by reference variable.
> var bytes = _malloc(size); // Allocate storage from emscripten
> heap.
> var err = _pn_data_format(this._data, bytes, sizeptr);
> var size = getValue(sizeptr, 'i32'); // Dereference the real size
> value;
>
> if (err === Module['Error']['OVERFLOW']) {
> _free(bytes);
> size *= 2;
> } else {
> var string = Pointer_stringify(bytes, size);
> _free(bytes);
> // Tidy up the memory that we allocated on emscripten's stack.
> Runtime.stackRestore(sp);
> this._check(err)
> return string;
> }
> }
> };
>
>
>
> So basically the "pattern" is to allocate some storage and call
> pn_data_format passing in the allocated storage and the size of the
> allocated storage - the latter is a pass by reference parameter.
>
> You need to check the error condition and if it's PN_OVERFLOW you need to
> free your original storage and try again with a bigger block - usual
> pattern is to double the size.
>
>
> I don't *think* there's any way to know a priori how much storage you are
> going to need because it's the call to pn_data_format that is actually
> "deserialising" the internal pn_data_t values into a pretty formatted form,
> so it doesn't actually know how big it's going to be until it has tried. In
> other words even if you could get the pointer data->str its value isn't
> useful until pn_data_format is actually being called - I think it's the
> private method pni_data_inspectify that starts that all off - if you look
> at the codec.c code you'll see:
>
> int pn_data_format(pn_data_t *data, char *bytes, size_t *size)
> {
> int err = pni_data_inspectify(data);
> if (err) return err;
> if (pn_string_size(data->str) >= *size) {
> return PN_OVERFLOW;
> } else {
> pn_string_put(data->str, bytes);
> *size = pn_string_size(data->str);
> return 0;
> }
> }
>
>
> and
>
> static int pni_data_inspectify(pn_data_t *data)
> {
> int err = pn_string_set(data->str, "");
> if (err) return err;
> return pn_data_inspect(data, data->str);
> }
>
> so inspectify initialises the internal string then calls inspect (which in
> turn calls pni_data_traverse which, as its name suggests crawls the
> internal data graph deserialising into the internal string).
>
> Having looked at the code I'm now pretty sure that it's not possible to do
> what you'd like and the only option is to check PN_OVERFLOW and realloc as
> necessary.
>
> Hope that helps,
> Frase
>
>
> On 13/02/15 21:22, Michael Ivanov wrote:
>
>> Hallo,
>>
>> I am trying to use proton routine for data formatting.
>> pn_data_format checks suire of provided buffer and in case it is not
>> enough just refuses to copy anything at all.
>>
>> Is it possible to find out how much does it need beforehand?
>>
>> Ideally would be if one had the possibility to get the pointer to this
>> internal string (data->str) which already holds the string representation
>> of data. Does this option exist?
>>
>> Best regards,
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: pn_data_format
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hello Mikhail,
I'm not an expert on pn_data_format and TBH I've never used it in pure C
code, only to write other language bindings. I don't know if it's
possible to find out beforehand. My guess is probably not - the python
binding (which is fairly canonical) looks like this:
def format(self):
sz = 16
while True:
err, result = pn_data_format(self._data, sz)
if err == PN_OVERFLOW:
sz *= 2
continue
else:
self._check(err)
return result
And I borrowed that pattern when I did the JavaScript binding, which
looks like this:
/**
* Format the encoded AMQP Data into a string representation and return it.
* @method format
* @memberof! proton.Data#
* @returns {string} a formatted string representation of the encoded Data.
*/
_Data_['format'] = function() {
var sp = Runtime.stackSave();
var sizeptr = allocate(4, 'i32', ALLOC_STACK);
var size = 1024; // Pass by reference variable - need to use
setValue to initialise it.
while (true) {
setValue(sizeptr, size, 'i32'); // Set pass by reference variable.
var bytes = _malloc(size); // Allocate storage from
emscripten heap.
var err = _pn_data_format(this._data, bytes, sizeptr);
var size = getValue(sizeptr, 'i32'); // Dereference the real
size value;
if (err === Module['Error']['OVERFLOW']) {
_free(bytes);
size *= 2;
} else {
var string = Pointer_stringify(bytes, size);
_free(bytes);
// Tidy up the memory that we allocated on emscripten's stack.
Runtime.stackRestore(sp);
this._check(err)
return string;
}
}
};
So basically the "pattern" is to allocate some storage and call
pn_data_format passing in the allocated storage and the size of the
allocated storage - the latter is a pass by reference parameter.
You need to check the error condition and if it's PN_OVERFLOW you need
to free your original storage and try again with a bigger block - usual
pattern is to double the size.
I don't *think* there's any way to know a priori how much storage you
are going to need because it's the call to pn_data_format that is
actually "deserialising" the internal pn_data_t values into a pretty
formatted form, so it doesn't actually know how big it's going to be
until it has tried. In other words even if you could get the pointer
data->str its value isn't useful until pn_data_format is actually being
called - I think it's the private method pni_data_inspectify that starts
that all off - if you look at the codec.c code you'll see:
int pn_data_format(pn_data_t *data, char *bytes, size_t *size)
{
int err = pni_data_inspectify(data);
if (err) return err;
if (pn_string_size(data->str) >= *size) {
return PN_OVERFLOW;
} else {
pn_string_put(data->str, bytes);
*size = pn_string_size(data->str);
return 0;
}
}
and
static int pni_data_inspectify(pn_data_t *data)
{
int err = pn_string_set(data->str, "");
if (err) return err;
return pn_data_inspect(data, data->str);
}
so inspectify initialises the internal string then calls inspect (which
in turn calls pni_data_traverse which, as its name suggests crawls the
internal data graph deserialising into the internal string).
Having looked at the code I'm now pretty sure that it's not possible to
do what you'd like and the only option is to check PN_OVERFLOW and
realloc as necessary.
Hope that helps,
Frase
On 13/02/15 21:22, Michael Ivanov wrote:
> Hallo,
>
> I am trying to use proton routine for data formatting.
> pn_data_format checks suire of provided buffer and in case it is not
> enough just refuses to copy anything at all.
>
> Is it possible to find out how much does it need beforehand?
>
> Ideally would be if one had the possibility to get the pointer to this
> internal string (data->str) which already holds the string representation
> of data. Does this option exist?
>
> Best regards,
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org