You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Ildar Absalyamov <il...@gmail.com> on 2016/06/16 22:27:00 UTC
Unsigned integers data types
Hi devs,
As I was generating various data distributions for statistics experiments one thing kept bothering me.
All Asterix integer types (int8, int16, int32, int64) are signed. However majority of real use cases does not require negative integer values. Seems like we are waisting half of the data range on something which does not get used that often. I guess part of the reason why we do that is because Java used to lack native support of unsigned integers. But since Java 8 there are methods which do unsigned comparison and division (summation, subtraction, multiplication are the same in both signed and unsigned cases). So it seems like conversion to support unsigned integers would not be that difficult.
Any thoughts on whether we need unsigned integers in the type system?
Best regards,
Ildar
Re: Unsigned integers data types
Posted by Mike Carey <dt...@gmail.com>.
There are some newer scalar data types that we support that might be
useful for at least some "unsigned integer use cases" - namely, the
binary type (which is like strings but is just bits).
On 6/16/16 4:10 PM, Ildar Absalyamov wrote:
> I guess my point was not to use unsigned integers int the case when int64 is not enough, but to allow using say uint32 instead of int64.
>
>> On Jun 16, 2016, at 16:03, Ian Maxon <im...@uci.edu> wrote:
>>
>> My 0.2c is that int64 is big enough if size was the consideration. Usually,
>> the times that I have wished I had unsigned integers in Java, were not
>> related to size constraints, but rather when I had to implement something
>> that required a lot of bitwise operations, since signedness makes that more
>> complicated. Usually int64 is big enough, and if it isn't, uint64 isn't
>> much better because you are probably representing something so huge that
>> arbitrary precision arithmetic is more appropriate.
>>
>> On Thu, Jun 16, 2016 at 3:47 PM, Ildar Absalyamov <
>> ildar.absalyamov@gmail.com> wrote:
>>
>>> Things like Spark and Flink don\u2019t do that as well, but because they need
>>> integration with proper Java types.
>>>
>>>> On Jun 16, 2016, at 15:45, Yingyi Bu <bu...@gmail.com> wrote:
>>>>
>>>>>> Is there any database or SQL implementation supporting that?
>>>> Ok, it turns out MySQL supports that, while Postgres, MS SQL and Hive do
>>>> not have that.
>>>>
>>>> Best,
>>>> Yingyi
>>>>
>>>> On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu <bu...@gmail.com> wrote:
>>>>
>>>>>>> I guess part of the reason why we do that is because Java used to lack
>>>>> native support of unsigned integers.
>>>>> Is there any database or SQL implementation supporting that?
>>>>>
>>>>> FYI:
>>>>>
>>>>>
>>> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
>>>>> Best,
>>>>> Yingyi
>>>>>
>>>>>
>>>>> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
>>>>> ildar.absalyamov@gmail.com> wrote:
>>>>>
>>>>>> Hi devs,
>>>>>>
>>>>>> As I was generating various data distributions for statistics
>>> experiments
>>>>>> one thing kept bothering me.
>>>>>> All Asterix integer types (int8, int16, int32, int64) are signed.
>>> However
>>>>>> majority of real use cases does not require negative integer values.
>>> Seems
>>>>>> like we are waisting half of the data range on something which does
>>> not get
>>>>>> used that often. I guess part of the reason why we do that is because
>>> Java
>>>>>> used to lack native support of unsigned integers. But since Java 8
>>> there
>>>>>> are methods which do unsigned comparison and division (summation,
>>>>>> subtraction, multiplication are the same in both signed and unsigned
>>>>>> cases). So it seems like conversion to support unsigned integers would
>>> not
>>>>>> be that difficult.
>>>>>>
>>>>>> Any thoughts on whether we need unsigned integers in the type system?
>>>>>>
>>>>>> Best regards,
>>>>>> Ildar
>>>>>>
>>>>>>
>>> Best regards,
>>> Ildar
>>>
>>>
> Best regards,
> Ildar
>
Re: Unsigned integers data types
Posted by Ildar Absalyamov <il...@gmail.com>.
I guess my point was not to use unsigned integers int the case when int64 is not enough, but to allow using say uint32 instead of int64.
> On Jun 16, 2016, at 16:03, Ian Maxon <im...@uci.edu> wrote:
>
> My 0.2c is that int64 is big enough if size was the consideration. Usually,
> the times that I have wished I had unsigned integers in Java, were not
> related to size constraints, but rather when I had to implement something
> that required a lot of bitwise operations, since signedness makes that more
> complicated. Usually int64 is big enough, and if it isn't, uint64 isn't
> much better because you are probably representing something so huge that
> arbitrary precision arithmetic is more appropriate.
>
> On Thu, Jun 16, 2016 at 3:47 PM, Ildar Absalyamov <
> ildar.absalyamov@gmail.com> wrote:
>
>> Things like Spark and Flink don’t do that as well, but because they need
>> integration with proper Java types.
>>
>>> On Jun 16, 2016, at 15:45, Yingyi Bu <bu...@gmail.com> wrote:
>>>
>>>>> Is there any database or SQL implementation supporting that?
>>> Ok, it turns out MySQL supports that, while Postgres, MS SQL and Hive do
>>> not have that.
>>>
>>> Best,
>>> Yingyi
>>>
>>> On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu <bu...@gmail.com> wrote:
>>>
>>>>>> I guess part of the reason why we do that is because Java used to lack
>>>> native support of unsigned integers.
>>>> Is there any database or SQL implementation supporting that?
>>>>
>>>> FYI:
>>>>
>>>>
>> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
>>>>
>>>> Best,
>>>> Yingyi
>>>>
>>>>
>>>> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
>>>> ildar.absalyamov@gmail.com> wrote:
>>>>
>>>>> Hi devs,
>>>>>
>>>>> As I was generating various data distributions for statistics
>> experiments
>>>>> one thing kept bothering me.
>>>>> All Asterix integer types (int8, int16, int32, int64) are signed.
>> However
>>>>> majority of real use cases does not require negative integer values.
>> Seems
>>>>> like we are waisting half of the data range on something which does
>> not get
>>>>> used that often. I guess part of the reason why we do that is because
>> Java
>>>>> used to lack native support of unsigned integers. But since Java 8
>> there
>>>>> are methods which do unsigned comparison and division (summation,
>>>>> subtraction, multiplication are the same in both signed and unsigned
>>>>> cases). So it seems like conversion to support unsigned integers would
>> not
>>>>> be that difficult.
>>>>>
>>>>> Any thoughts on whether we need unsigned integers in the type system?
>>>>>
>>>>> Best regards,
>>>>> Ildar
>>>>>
>>>>>
>>>>
>>
>> Best regards,
>> Ildar
>>
>>
Best regards,
Ildar
Re: Unsigned integers data types
Posted by Ian Maxon <im...@uci.edu>.
My 0.2c is that int64 is big enough if size was the consideration. Usually,
the times that I have wished I had unsigned integers in Java, were not
related to size constraints, but rather when I had to implement something
that required a lot of bitwise operations, since signedness makes that more
complicated. Usually int64 is big enough, and if it isn't, uint64 isn't
much better because you are probably representing something so huge that
arbitrary precision arithmetic is more appropriate.
On Thu, Jun 16, 2016 at 3:47 PM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:
> Things like Spark and Flink don’t do that as well, but because they need
> integration with proper Java types.
>
> > On Jun 16, 2016, at 15:45, Yingyi Bu <bu...@gmail.com> wrote:
> >
> >>> Is there any database or SQL implementation supporting that?
> > Ok, it turns out MySQL supports that, while Postgres, MS SQL and Hive do
> > not have that.
> >
> > Best,
> > Yingyi
> >
> > On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu <bu...@gmail.com> wrote:
> >
> >>>> I guess part of the reason why we do that is because Java used to lack
> >> native support of unsigned integers.
> >> Is there any database or SQL implementation supporting that?
> >>
> >> FYI:
> >>
> >>
> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
> >>
> >> Best,
> >> Yingyi
> >>
> >>
> >> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
> >> ildar.absalyamov@gmail.com> wrote:
> >>
> >>> Hi devs,
> >>>
> >>> As I was generating various data distributions for statistics
> experiments
> >>> one thing kept bothering me.
> >>> All Asterix integer types (int8, int16, int32, int64) are signed.
> However
> >>> majority of real use cases does not require negative integer values.
> Seems
> >>> like we are waisting half of the data range on something which does
> not get
> >>> used that often. I guess part of the reason why we do that is because
> Java
> >>> used to lack native support of unsigned integers. But since Java 8
> there
> >>> are methods which do unsigned comparison and division (summation,
> >>> subtraction, multiplication are the same in both signed and unsigned
> >>> cases). So it seems like conversion to support unsigned integers would
> not
> >>> be that difficult.
> >>>
> >>> Any thoughts on whether we need unsigned integers in the type system?
> >>>
> >>> Best regards,
> >>> Ildar
> >>>
> >>>
> >>
>
> Best regards,
> Ildar
>
>
Re: Unsigned integers data types
Posted by Ildar Absalyamov <il...@gmail.com>.
Things like Spark and Flink don’t do that as well, but because they need integration with proper Java types.
> On Jun 16, 2016, at 15:45, Yingyi Bu <bu...@gmail.com> wrote:
>
>>> Is there any database or SQL implementation supporting that?
> Ok, it turns out MySQL supports that, while Postgres, MS SQL and Hive do
> not have that.
>
> Best,
> Yingyi
>
> On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu <bu...@gmail.com> wrote:
>
>>>> I guess part of the reason why we do that is because Java used to lack
>> native support of unsigned integers.
>> Is there any database or SQL implementation supporting that?
>>
>> FYI:
>>
>> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
>>
>> Best,
>> Yingyi
>>
>>
>> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
>> ildar.absalyamov@gmail.com> wrote:
>>
>>> Hi devs,
>>>
>>> As I was generating various data distributions for statistics experiments
>>> one thing kept bothering me.
>>> All Asterix integer types (int8, int16, int32, int64) are signed. However
>>> majority of real use cases does not require negative integer values. Seems
>>> like we are waisting half of the data range on something which does not get
>>> used that often. I guess part of the reason why we do that is because Java
>>> used to lack native support of unsigned integers. But since Java 8 there
>>> are methods which do unsigned comparison and division (summation,
>>> subtraction, multiplication are the same in both signed and unsigned
>>> cases). So it seems like conversion to support unsigned integers would not
>>> be that difficult.
>>>
>>> Any thoughts on whether we need unsigned integers in the type system?
>>>
>>> Best regards,
>>> Ildar
>>>
>>>
>>
Best regards,
Ildar
Re: Unsigned integers data types
Posted by Yingyi Bu <bu...@gmail.com>.
>> Is there any database or SQL implementation supporting that?
Ok, it turns out MySQL supports that, while Postgres, MS SQL and Hive do
not have that.
Best,
Yingyi
On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu <bu...@gmail.com> wrote:
> >> I guess part of the reason why we do that is because Java used to lack
> native support of unsigned integers.
> Is there any database or SQL implementation supporting that?
>
> FYI:
>
> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
>
> Best,
> Yingyi
>
>
> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
> ildar.absalyamov@gmail.com> wrote:
>
>> Hi devs,
>>
>> As I was generating various data distributions for statistics experiments
>> one thing kept bothering me.
>> All Asterix integer types (int8, int16, int32, int64) are signed. However
>> majority of real use cases does not require negative integer values. Seems
>> like we are waisting half of the data range on something which does not get
>> used that often. I guess part of the reason why we do that is because Java
>> used to lack native support of unsigned integers. But since Java 8 there
>> are methods which do unsigned comparison and division (summation,
>> subtraction, multiplication are the same in both signed and unsigned
>> cases). So it seems like conversion to support unsigned integers would not
>> be that difficult.
>>
>> Any thoughts on whether we need unsigned integers in the type system?
>>
>> Best regards,
>> Ildar
>>
>>
>
Re: Unsigned integers data types
Posted by Yingyi Bu <bu...@gmail.com>.
>> I guess part of the reason why we do that is because Java used to lack
native support of unsigned integers.
Is there any database or SQL implementation supporting that?
FYI:
http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
Best,
Yingyi
On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:
> Hi devs,
>
> As I was generating various data distributions for statistics experiments
> one thing kept bothering me.
> All Asterix integer types (int8, int16, int32, int64) are signed. However
> majority of real use cases does not require negative integer values. Seems
> like we are waisting half of the data range on something which does not get
> used that often. I guess part of the reason why we do that is because Java
> used to lack native support of unsigned integers. But since Java 8 there
> are methods which do unsigned comparison and division (summation,
> subtraction, multiplication are the same in both signed and unsigned
> cases). So it seems like conversion to support unsigned integers would not
> be that difficult.
>
> Any thoughts on whether we need unsigned integers in the type system?
>
> Best regards,
> Ildar
>
>