You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by John Plevyak <jp...@acm.org> on 2010/05/19 02:52:51 UTC
stdint.h
It seems that g++/C++ is completely broken with respect to portable
64-bit numbers and scanf/printf.
uint64_t is long long int on 32-bit machines and long int on 64-bit
machines and scanf/printf require the corresponding %lld %ld which is
painfully difficult.
Until they fix it, either we use the pain in the ass PRIu64 macro "x = "
PRIu64 " things" which screws up computed print strings and user
configurable strings or we suck it up and continue to use tsu64 for
64bit numbers or switch to int64 uint64. I don't think tsu64 is
readable. At least inku64 scanned as intu64. tsu64 scans as some
aol username.
My suggestion is to go with int64, uint64. It is readable, generic,
doesn't conflict with standard libs and lets us use:
long long int which is portable for uint64
int which is portable for int32
short which is portable for int16
Why the hell uint64_t is defined this way and g++ complains even
though long int == long long it is beyond me.
Suggestions, comments?
john
Re: stdint.h please vote
Posted by Jason <ja...@gmail.com>.
+1, against type casting for this
On Wed, May 19, 2010 at 12:31 PM, Bryan Call <bc...@yahoo-inc.com> wrote:
> On 05/19/2010 09:14 AM, John Plevyak wrote:
>
>> Yes we will change INXXX_MAX etc.
>>
>> Unfortunately int64 is not standard. So we have a couple alternatives.
>>
>> 1) use nonstandard types where
>>
>> typedef long long int int64;
>> typedef int int32;
>> typedef short int16;
>> typedef char int8;
>>
>> etc.
>>
>> which work on every system and which permit seamless printf
>> using %lld and %d. This is the way we do it now.
>>
>>
>>
>
> +1 - This is the lesser of the evils...
>
> Only because there is no other way to get *printf() from complaining
> without using macros or type casting.
>
>
> 2. use "standard" types and cast all printf/scanf calls:
>>
>> printf("x = %lld", *(long long int*)&my_standard_uint64_t_var)
>>
>> 3. use the "standard" types and the "standard" macros:
>>
>> printf("x = " PRIi64, my_standard_uint64_t_var)
>>
>> however, it should be noted that this doesn't work for strings which are
>> not constants, are read from a configuration file or input by the
>> user, computed etc.
>>
>> I think that 1 is the least of the evils. 2 is dangerous and ugly
>> and 3 is ugly and a pain, and potentially complex as it might mean
>> rewriting printf/scanf strings on the fly and this standard is
>> relatively recent which means that lots of older systems don't support
>> it which means including the headers ourselves. MS Studio didn't
>> support it until the 2010 version for example.
>>
>>
>> On 5/19/2010 8:16 AM, Leif Hedstrom wrote:
>>
>>
>>> On 05/19/2010 08:23 AM, John Plevyak wrote:
>>>
>>>
>>>> I propose that we change
>>>>
>>>> ink64 -> int64
>>>> inku64 -> uint64
>>>> ink32 -> int32
>>>> inku32 -> uint32
>>>> ink16 -> int16
>>>> inku16 -> uint16
>>>> ink8 -> int8
>>>> inku8 -> uint8
>>>>
>>>> because
>>>>
>>>> 1) we decided to move from ink -> ts
>>>> 2) tsu64 doesn't scan like an integer
>>>> 3) int64_t is long on linux which is incompatible with %lld the
>>>> only standard and universally compatible way to read/write a
>>>> 64-bit number
>>>> 4) int64 is similar but not the same as ink64_t and it scans well
>>>> 5) I tried it and it works!
>>>>
>>>>
>>>>
>>> Would this be a name change only (in our ink_port.h file)? Or do we pull
>>> in int64 etc from some standard (ANSI / POSIX include file)? My stdint.h
>>> only has the int64_t etc. definitions.
>>>
>>> Also, if we do this, shouldn't we also change
>>>
>>> #define INKU64_MAX (18446744073709551615ULL)
>>> #define INK64_MAX (9223372036854775807LL)
>>> #define INK64_MIN (-INK64_MAX -1LL)
>>> #define INKU32_MAX (4294967295U)
>>> #define INK32_MAX (2147483647)
>>> #define INK32_MIN (-2147483647-1)
>>>
>>>
>>> I believe there are similar defines available in stdint.h. (Fwiw, my
>>> 64-bit changes in the HttpSM uses INK64_MAX in various places).
>>>
>>> That much said, +1 on eliminating our own INK defines, and use standard
>>> definitions.
>>>
>>> -- leif
>>>
>>>
>>
>>
>
>
>
Re: stdint.h please vote
Posted by George Paul <ge...@apache.org>.
Late but
+1
-George
On 5/19/10 9:57 AM, Leif Hedstrom wrote:
> On 05/19/2010 10:31 AM, Bryan Call wrote:
>> On 05/19/2010 09:14 AM, John Plevyak wrote:
>>> Yes we will change INXXX_MAX etc.
>>>
>>> Unfortunately int64 is not standard. So we have a couple alternatives.
>>>
>>> 1) use nonstandard types where
>>>
>>> typedef long long int int64;
>>> typedef int int32;
>>> typedef short int16;
>>> typedef char int8;
>>>
>>> etc.
>>>
>>> which work on every system and which permit seamless printf
>>> using %lld and %d. This is the way we do it now.
>>>
>>
>> +1 - This is the lesser of the evils...
>
>
> +1, less evil == good.
>
> -- leif
>
Re: stdint.h please vote
Posted by Leif Hedstrom <zw...@apache.org>.
On 05/19/2010 10:31 AM, Bryan Call wrote:
> On 05/19/2010 09:14 AM, John Plevyak wrote:
>> Yes we will change INXXX_MAX etc.
>>
>> Unfortunately int64 is not standard. So we have a couple alternatives.
>>
>> 1) use nonstandard types where
>>
>> typedef long long int int64;
>> typedef int int32;
>> typedef short int16;
>> typedef char int8;
>>
>> etc.
>>
>> which work on every system and which permit seamless printf
>> using %lld and %d. This is the way we do it now.
>>
>
> +1 - This is the lesser of the evils...
+1, less evil == good.
-- leif
Re: stdint.h please vote
Posted by "Alan M. Carroll" <am...@network-geographics.com>.
+1
Wednesday, May 19, 2010, 11:31:50 AM, you wrote:
> On 05/19/2010 09:14 AM, John Plevyak wrote:
>> Yes we will change INXXX_MAX etc.
>>
>> Unfortunately int64 is not standard. So we have a couple alternatives.
>>
>> 1) use nonstandard types where
>>
>> typedef long long int int64;
>> typedef int int32;
>> typedef short int16;
>> typedef char int8;
>>
>> etc.
>>
>> which work on every system and which permit seamless printf
>> using %lld and %d. This is the way we do it now.
Re: stdint.h please vote
Posted by Bryan Call <bc...@yahoo-inc.com>.
On 05/19/2010 09:14 AM, John Plevyak wrote:
> Yes we will change INXXX_MAX etc.
>
> Unfortunately int64 is not standard. So we have a couple alternatives.
>
> 1) use nonstandard types where
>
> typedef long long int int64;
> typedef int int32;
> typedef short int16;
> typedef char int8;
>
> etc.
>
> which work on every system and which permit seamless printf
> using %lld and %d. This is the way we do it now.
>
>
+1 - This is the lesser of the evils...
Only because there is no other way to get *printf() from complaining
without using macros or type casting.
> 2. use "standard" types and cast all printf/scanf calls:
>
> printf("x = %lld", *(long long int*)&my_standard_uint64_t_var)
>
> 3. use the "standard" types and the "standard" macros:
>
> printf("x = " PRIi64, my_standard_uint64_t_var)
>
> however, it should be noted that this doesn't work for strings which are
> not constants, are read from a configuration file or input by the
> user, computed etc.
>
> I think that 1 is the least of the evils. 2 is dangerous and ugly
> and 3 is ugly and a pain, and potentially complex as it might mean
> rewriting printf/scanf strings on the fly and this standard is
> relatively recent which means that lots of older systems don't support
> it which means including the headers ourselves. MS Studio didn't
> support it until the 2010 version for example.
>
>
> On 5/19/2010 8:16 AM, Leif Hedstrom wrote:
>
>> On 05/19/2010 08:23 AM, John Plevyak wrote:
>>
>>> I propose that we change
>>>
>>> ink64 -> int64
>>> inku64 -> uint64
>>> ink32 -> int32
>>> inku32 -> uint32
>>> ink16 -> int16
>>> inku16 -> uint16
>>> ink8 -> int8
>>> inku8 -> uint8
>>>
>>> because
>>>
>>> 1) we decided to move from ink -> ts
>>> 2) tsu64 doesn't scan like an integer
>>> 3) int64_t is long on linux which is incompatible with %lld the
>>> only standard and universally compatible way to read/write a
>>> 64-bit number
>>> 4) int64 is similar but not the same as ink64_t and it scans well
>>> 5) I tried it and it works!
>>>
>>>
>> Would this be a name change only (in our ink_port.h file)? Or do we pull
>> in int64 etc from some standard (ANSI / POSIX include file)? My stdint.h
>> only has the int64_t etc. definitions.
>>
>> Also, if we do this, shouldn't we also change
>>
>> #define INKU64_MAX (18446744073709551615ULL)
>> #define INK64_MAX (9223372036854775807LL)
>> #define INK64_MIN (-INK64_MAX -1LL)
>> #define INKU32_MAX (4294967295U)
>> #define INK32_MAX (2147483647)
>> #define INK32_MIN (-2147483647-1)
>>
>>
>> I believe there are similar defines available in stdint.h. (Fwiw, my
>> 64-bit changes in the HttpSM uses INK64_MAX in various places).
>>
>> That much said, +1 on eliminating our own INK defines, and use standard
>> definitions.
>>
>> -- leif
>>
>
Re: stdint.h please vote
Posted by John Plevyak <jp...@acm.org>.
Yes we will change INXXX_MAX etc.
Unfortunately int64 is not standard. So we have a couple alternatives.
1) use nonstandard types where
typedef long long int int64;
typedef int int32;
typedef short int16;
typedef char int8;
etc.
which work on every system and which permit seamless printf
using %lld and %d. This is the way we do it now.
2. use "standard" types and cast all printf/scanf calls:
printf("x = %lld", *(long long int*)&my_standard_uint64_t_var)
3. use the "standard" types and the "standard" macros:
printf("x = " PRIi64, my_standard_uint64_t_var)
however, it should be noted that this doesn't work for strings which are
not constants, are read from a configuration file or input by the
user, computed etc.
I think that 1 is the least of the evils. 2 is dangerous and ugly
and 3 is ugly and a pain, and potentially complex as it might mean
rewriting printf/scanf strings on the fly and this standard is
relatively recent which means that lots of older systems don't support
it which means including the headers ourselves. MS Studio didn't
support it until the 2010 version for example.
On 5/19/2010 8:16 AM, Leif Hedstrom wrote:
> On 05/19/2010 08:23 AM, John Plevyak wrote:
>> I propose that we change
>>
>> ink64 -> int64
>> inku64 -> uint64
>> ink32 -> int32
>> inku32 -> uint32
>> ink16 -> int16
>> inku16 -> uint16
>> ink8 -> int8
>> inku8 -> uint8
>>
>> because
>>
>> 1) we decided to move from ink -> ts
>> 2) tsu64 doesn't scan like an integer
>> 3) int64_t is long on linux which is incompatible with %lld the
>> only standard and universally compatible way to read/write a
>> 64-bit number
>> 4) int64 is similar but not the same as ink64_t and it scans well
>> 5) I tried it and it works!
>>
>
> Would this be a name change only (in our ink_port.h file)? Or do we pull
> in int64 etc from some standard (ANSI / POSIX include file)? My stdint.h
> only has the int64_t etc. definitions.
>
> Also, if we do this, shouldn't we also change
>
> #define INKU64_MAX (18446744073709551615ULL)
> #define INK64_MAX (9223372036854775807LL)
> #define INK64_MIN (-INK64_MAX -1LL)
> #define INKU32_MAX (4294967295U)
> #define INK32_MAX (2147483647)
> #define INK32_MIN (-2147483647-1)
>
>
> I believe there are similar defines available in stdint.h. (Fwiw, my
> 64-bit changes in the HttpSM uses INK64_MAX in various places).
>
> That much said, +1 on eliminating our own INK defines, and use standard
> definitions.
>
> -- leif
Re: stdint.h please vote
Posted by Leif Hedstrom <zw...@apache.org>.
On 05/19/2010 08:23 AM, John Plevyak wrote:
> I propose that we change
>
> ink64 -> int64
> inku64 -> uint64
> ink32 -> int32
> inku32 -> uint32
> ink16 -> int16
> inku16 -> uint16
> ink8 -> int8
> inku8 -> uint8
>
> because
>
> 1) we decided to move from ink -> ts
> 2) tsu64 doesn't scan like an integer
> 3) int64_t is long on linux which is incompatible with %lld the
> only standard and universally compatible way to read/write a 64-bit number
> 4) int64 is similar but not the same as ink64_t and it scans well
> 5) I tried it and it works!
>
Would this be a name change only (in our ink_port.h file)? Or do we pull
in int64 etc from some standard (ANSI / POSIX include file)? My stdint.h
only has the int64_t etc. definitions.
Also, if we do this, shouldn't we also change
#define INKU64_MAX (18446744073709551615ULL)
#define INK64_MAX (9223372036854775807LL)
#define INK64_MIN (-INK64_MAX -1LL)
#define INKU32_MAX (4294967295U)
#define INK32_MAX (2147483647)
#define INK32_MIN (-2147483647-1)
I believe there are similar defines available in stdint.h. (Fwiw, my
64-bit changes in the HttpSM uses INK64_MAX in various places).
That much said, +1 on eliminating our own INK defines, and use standard
definitions.
-- leif
Re: stdint.h please vote
Posted by John Plevyak <jp...@acm.org>.
I propose that we change
ink64 -> int64
inku64 -> uint64
ink32 -> int32
inku32 -> uint32
ink16 -> int16
inku16 -> uint16
ink8 -> int8
inku8 -> uint8
because
1) we decided to move from ink -> ts
2) tsu64 doesn't scan like an integer
3) int64_t is long on linux which is incompatible with %lld the
only standard and universally compatible way to read/write a 64-bit number
4) int64 is similar but not the same as ink64_t and it scans well
5) I tried it and it works!
john
On 5/19/2010 7:06 AM, Leif Hedstrom wrote:
> On 05/19/2010 04:17 AM, Mladen Turk wrote:
>> On 05/19/2010 08:25 AM, John Plevyak wrote:
>>>
>>> I hate to say it, but the simplest thing is
>>> to use long long int for all this junk. In practice
>>> that is what it is on most systems, or at
>>> least that covers their useful range.
>>>
>>
>> Agreed.
>> Think that Leif is working on 64-bit file API, and
>> forcing that for both 32 and 64 bit systems would
>> certainly simplify the need for casting.
>
> Yeah. All my changes are using ink64 though, and %lld when using one of
> the formatted output APIs. I figured it'd be easy to change that later
> if/when we decide to change types.
>
> -- Leif
Re: stdint.h
Posted by Leif Hedstrom <zw...@apache.org>.
On 05/19/2010 04:17 AM, Mladen Turk wrote:
> On 05/19/2010 08:25 AM, John Plevyak wrote:
>>
>> I hate to say it, but the simplest thing is
>> to use long long int for all this junk. In practice
>> that is what it is on most systems, or at
>> least that covers their useful range.
>>
>
> Agreed.
> Think that Leif is working on 64-bit file API, and
> forcing that for both 32 and 64 bit systems would
> certainly simplify the need for casting.
Yeah. All my changes are using ink64 though, and %lld when using one of
the formatted output APIs. I figured it'd be easy to change that later
if/when we decide to change types.
-- Leif
Re: stdint.h
Posted by Mladen Turk <mt...@apache.org>.
On 05/19/2010 08:25 AM, John Plevyak wrote:
>
> I hate to say it, but the simplest thing is
> to use long long int for all this junk. In practice
> that is what it is on most systems, or at
> least that covers their useful range.
>
Agreed.
Think that Leif is working on 64-bit file API, and
forcing that for both 32 and 64 bit systems would
certainly simplify the need for casting.
>
> Given that traffic server has (or at least had)
> exactly zero mallocs on the critical path and
> nevertheless the freelist allocator is the top profiled
> function make it abundantly clear that cout is not a rational
> answer for high performance systems.
>
Sure, there's nothing wrong with printf family of functions.
Regards
--
^TM
Re: stdint.h
Posted by John Plevyak <jp...@acm.org>.
I hate to say it, but the simplest thing is
to use long long int for all this junk. In practice
that is what it is on most systems, or at
least that covers their useful range.
The unfortunate answer is that C/C++ is just
plain ill considered. The argument that you
should just use cout is ludicrous as a typical
cout implementation has 17 hidden mallocs just
to output a simple line with a variable in it
(see the fastformat site).
Given that traffic server has (or at least had)
exactly zero mallocs on the critical path and
nevertheless the freelist allocator is the top profiled
function make it abundantly clear that cout is not a rational
answer for high performance systems.
So that leaves us with the lowest common demoninator
which is the standard 64-bit number which is long long int
(a stupid name if there ever was one).
As they say about political systems, C/C++ is the worst
language... except for all the others.
john
On 5/18/2010 10:23 PM, Mladen Turk wrote:
> On 05/19/2010 07:02 AM, John Plevyak wrote:
>>
>> I would say that if we can't find a system which doesn't
>> support the standard %lld for 64-bit numbers then we should
>> just go with the standard. It is simpler and it will only
>> be more right as time passes because it is the standard.
>>
>
> Fair enough. Makes sense since we are not supporting
> such waste majority of platforms and compilers like APR.
>
> However how about const numeric macros
> (123UL and 123ULL for example)
> Are they safe enough for using them both with uint64_t and inku64?
>
> And I suppose we'll have to cast the ink_off_t to ink64
> all the times.
> There's also a pid_t format error problem, but I
> suppose we don't use that so widely.
>
> What about ssize_t and size_t? This can be either
> %d, %ld and %u or %lu AFAICT
>
>
> Regards
Re: stdint.h
Posted by Mladen Turk <mt...@apache.org>.
On 05/19/2010 07:02 AM, John Plevyak wrote:
>
> I would say that if we can't find a system which doesn't
> support the standard %lld for 64-bit numbers then we should
> just go with the standard. It is simpler and it will only
> be more right as time passes because it is the standard.
>
Fair enough. Makes sense since we are not supporting
such waste majority of platforms and compilers like APR.
However how about const numeric macros
(123UL and 123ULL for example)
Are they safe enough for using them both with uint64_t and inku64?
And I suppose we'll have to cast the ink_off_t to ink64
all the times.
There's also a pid_t format error problem, but I
suppose we don't use that so widely.
What about ssize_t and size_t? This can be either
%d, %ld and %u or %lu AFAICT
Regards
--
^TM
Re: stdint.h
Posted by John Plevyak <jp...@acm.org>.
All compilers that are C99 compliant must accept
%lld as long long int where sizeof(long long int) == 8.
Sun 64 and linux 64 both have sizeof(long) == 8, but
because of C99 they also have sizeof(long long int) == 8
and %ld and %lld will both work with 64-bit arguments,
however gcc/g++ will complain if you pass a long to something
expecting %lld even if the are the same size because it is
pedantic. This is the problem with uint64_t, because they
choose to use "long" instead of "long long int" despite
the fact that they are both the same size.
Windows Studio 2003 did not support %lld, but as of
Studio 2005 it now does. The %I64d, like %q on FreeBSD,
is a holdover from when there was not standard solution.
I would say that if we can't find a system which doesn't
support the standard %lld for 64-bit numbers then we should
just go with the standard. It is simpler and it will only
be more right as time passes because it is the standard.
john
On 5/18/2010 9:50 PM, Mladen Turk wrote:
> On 05/19/2010 06:35 AM, John Plevyak wrote:
>>
>>
>> However, since the C99 and new C++ standard declares that long long int
>> is 64-bit and since just about everyone already has that:
>> linux, solaris, freebsd, mac, and since I have been able to use it
>> portably for several years now I would rather go that route.
>>
>
> If we ever plan to port TS to windows (or at least some parts
> of it, then there's %I64d format, next if sizeof(long) == 8, then
> the correct format is %ld not %lld, etc.
> Think that some compiles complain if you try to use %lld instead
> %ld in such situations.
>
>
>
> Regards
Re: stdint.h
Posted by Mladen Turk <mt...@apache.org>.
On 05/19/2010 06:35 AM, John Plevyak wrote:
>
>
> However, since the C99 and new C++ standard declares that long long int
> is 64-bit and since just about everyone already has that:
> linux, solaris, freebsd, mac, and since I have been able to use it
> portably for several years now I would rather go that route.
>
If we ever plan to port TS to windows (or at least some parts
of it, then there's %I64d format, next if sizeof(long) == 8, then
the correct format is %ld not %lld, etc.
Think that some compiles complain if you try to use %lld instead
%ld in such situations.
Regards
--
^TM
Re: stdint.h
Posted by John Plevyak <jp...@acm.org>.
There are standard macros for doing that (e.g. PRIu64) which are defined
in /usr/include/inttypes.h for ISO C99 7.8.
If we had to go that route I would go with those rather than a
roll-your-own even if that means defining them on platforms that
don't have ISC C99 support yet (only a decade past).
However, since the C99 and new C++ standard declares that long long int
is 64-bit and since just about everyone already has that:
linux, solaris, freebsd, mac, and since I have been able to use it
portably for several years now I would rather go that route.
If there is a platform which doesn't support long long int as 64-bit
I don't doubt that it supports 64-bit at all and I doubt we will
every be called upon to support it, and given that it is the new
standard, if we ignore the problem it will, very definitely, go away.
john
On 5/18/2010 9:15 PM, Mladen Turk wrote:
> On 05/19/2010 02:52 AM, John Plevyak wrote:
>>
>>
>> Why the hell uint64_t is defined this way and g++ complains even
>> though long int == long long it is beyond me.
>>
>> Suggestions, comments?
>>
>
> I saw whole bunch of %ld, %lld stuff in the code which will IMHO
> never be portable no matter what type you use.
>
> With APR (and some other projects) we detect those types
> at configure time, and define printf/scanf macros
> eg. apr_int64_t, apr_uint64_t and then have
> APR_INT64_FMT_T, and APR_UINT64_FMT_T
>
> This makes printf/scanf code a little bit longer:
> eg. printf("Use some %" APR_UINT64_FMT_T " 64-bit integer\n",
> (apr_uint64_t)foo);
> but it is portable, especially between 32 and 64 bit compilers
>
> Think we should use something like that, because we already
> have the detection code (from other projects) and it works well
> for a variety of different compilers.
>
> At the end it's more or less discipline and search/replace,
> and I was going to add them anyhow.
>
>
> Regards
Re: stdint.h
Posted by Mladen Turk <mt...@apache.org>.
On 05/19/2010 02:52 AM, John Plevyak wrote:
>
>
> Why the hell uint64_t is defined this way and g++ complains even
> though long int == long long it is beyond me.
>
> Suggestions, comments?
>
I saw whole bunch of %ld, %lld stuff in the code which will IMHO
never be portable no matter what type you use.
With APR (and some other projects) we detect those types
at configure time, and define printf/scanf macros
eg. apr_int64_t, apr_uint64_t and then have
APR_INT64_FMT_T, and APR_UINT64_FMT_T
This makes printf/scanf code a little bit longer:
eg. printf("Use some %" APR_UINT64_FMT_T " 64-bit integer\n", (apr_uint64_t)foo);
but it is portable, especially between 32 and 64 bit compilers
Think we should use something like that, because we already
have the detection code (from other projects) and it works well
for a variety of different compilers.
At the end it's more or less discipline and search/replace,
and I was going to add them anyhow.
Regards
--
^TM