You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Anitha Thankappan <an...@quantiphi.com> on 2022/04/05 13:40:27 UTC
BIGDECIMAL data handling
Hi,
I am doing a developed a Fink connector for GCP BigQuery.
When we are reading BIGNUMERIC data from BigQuery table, we didn't find
any matching data types in Flink. Also failed to implement user defined
data type for BIGNUMERIC.
Please let me know if there is any way to handle this.
Thanks in Advance,
Anitha Thankappan
--
_This message contains information that may be privileged or confidential
and is the property of the Quantiphi Inc and/or its affiliates_. It is
intended only for the person to whom it is addressed. _If you are not the
intended recipient, any review, dissemination, distribution, copying,
storage or other use of all or any portion of this message is strictly
prohibited. If you received this message in error, please immediately
notify the sender by reply e-mail and delete this message in its
*entirety*___
Re: BIGDECIMAL data handling
Posted by Francesco Guardiani <fr...@ververica.com>.
Is there any reason for not using DECIMAL provided by Flink SQL?
On Tue, Apr 5, 2022 at 4:06 PM Anitha Thankappan <
anitha.thankappan@quantiphi.com> wrote:
> Hi Martijn,
>
> I am using flink version 1.11.0.
> The flink application code snippet is like:
>
> [image: image.png]
>
> The Error I am receiving while providing BIGDECIMAL as datatype is :
> [image: image.png]
>
> Can I use unregistered structured custom data types in DDLs like Create
> Table?
>
> Thanks and Regards,
> Anitha Thankappan
>
>
> On Tue, Apr 5, 2022 at 7:21 PM Martijn Visser <ma...@apache.org>
> wrote:
>
>> Hi Anitha,
>>
>> Looking at Bigquery's documentation, they're aliasing it as a BIGDECIMAL
>> [1]. According to Flink's documentation, you can create an unregistered
>> structured type as an user-defined data type [2]. You're mentioning that
>> you've failed to implement this, but what is the issue that you're running
>> into? Bigdecimal is mentioned in Flink's documentation.
>>
>> Best regards,
>>
>> Martijn Visser
>> https://twitter.com/MartijnVisser82
>> https://github.com/MartijnVisser
>>
>> [1]
>> https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types
>> [2]
>> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/types/#user-defined-data-types
>>
>>
>> On Tue, 5 Apr 2022 at 15:41, Anitha Thankappan <
>> anitha.thankappan@quantiphi.com> wrote:
>>
>>> Hi,
>>>
>>> I am doing a developed a Fink connector for GCP BigQuery.
>>> When we are reading BIGNUMERIC data from BigQuery table, we
>>> didn't find any matching data types in Flink. Also failed to implement user
>>> defined data type for BIGNUMERIC.
>>>
>>> Please let me know if there is any way to handle this.
>>>
>>> Thanks in Advance,
>>> Anitha Thankappan
>>>
>>> *This message contains information that may be privileged or
>>> confidential and is the property of the Quantiphi Inc and/or its affiliates**.
>>> It is intended only for the person to whom it is addressed. **If you
>>> are not the intended recipient, any review, dissemination, distribution,
>>> copying, storage or other use of all or any portion of this message is
>>> strictly prohibited. If you received this message in error, please
>>> immediately notify the sender by reply e-mail and delete this message in
>>> its **entirety*
>>>
>>
> *This message contains information that may be privileged or confidential
> and is the property of the Quantiphi Inc and/or its affiliates**. It is
> intended only for the person to whom it is addressed. **If you are not
> the intended recipient, any review, dissemination, distribution, copying,
> storage or other use of all or any portion of this message is strictly
> prohibited. If you received this message in error, please immediately
> notify the sender by reply e-mail and delete this message in its *
> *entirety*
>
Re: BIGDECIMAL data handling
Posted by Anitha Thankappan <an...@quantiphi.com>.
Hi Martijn,
I am using flink version 1.11.0.
The flink application code snippet is like:
[image: image.png]
The Error I am receiving while providing BIGDECIMAL as datatype is :
[image: image.png]
Can I use unregistered structured custom data types in DDLs like Create
Table?
Thanks and Regards,
Anitha Thankappan
On Tue, Apr 5, 2022 at 7:21 PM Martijn Visser <ma...@apache.org>
wrote:
> Hi Anitha,
>
> Looking at Bigquery's documentation, they're aliasing it as a BIGDECIMAL
> [1]. According to Flink's documentation, you can create an unregistered
> structured type as an user-defined data type [2]. You're mentioning that
> you've failed to implement this, but what is the issue that you're running
> into? Bigdecimal is mentioned in Flink's documentation.
>
> Best regards,
>
> Martijn Visser
> https://twitter.com/MartijnVisser82
> https://github.com/MartijnVisser
>
> [1]
> https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types
> [2]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/types/#user-defined-data-types
>
>
> On Tue, 5 Apr 2022 at 15:41, Anitha Thankappan <
> anitha.thankappan@quantiphi.com> wrote:
>
>> Hi,
>>
>> I am doing a developed a Fink connector for GCP BigQuery.
>> When we are reading BIGNUMERIC data from BigQuery table, we didn't find
>> any matching data types in Flink. Also failed to implement user defined
>> data type for BIGNUMERIC.
>>
>> Please let me know if there is any way to handle this.
>>
>> Thanks in Advance,
>> Anitha Thankappan
>>
>> *This message contains information that may be privileged or confidential
>> and is the property of the Quantiphi Inc and/or its affiliates**. It is
>> intended only for the person to whom it is addressed. **If you are not
>> the intended recipient, any review, dissemination, distribution, copying,
>> storage or other use of all or any portion of this message is strictly
>> prohibited. If you received this message in error, please immediately
>> notify the sender by reply e-mail and delete this message in its *
>> *entirety*
>>
>
--
_This message contains information that may be privileged or confidential
and is the property of the Quantiphi Inc and/or its affiliates_. It is
intended only for the person to whom it is addressed. _If you are not the
intended recipient, any review, dissemination, distribution, copying,
storage or other use of all or any portion of this message is strictly
prohibited. If you received this message in error, please immediately
notify the sender by reply e-mail and delete this message in its
*entirety*___
Re: BIGDECIMAL data handling
Posted by Martijn Visser <ma...@apache.org>.
Hi Anitha,
Looking at Bigquery's documentation, they're aliasing it as a BIGDECIMAL
[1]. According to Flink's documentation, you can create an unregistered
structured type as an user-defined data type [2]. You're mentioning that
you've failed to implement this, but what is the issue that you're running
into? Bigdecimal is mentioned in Flink's documentation.
Best regards,
Martijn Visser
https://twitter.com/MartijnVisser82
https://github.com/MartijnVisser
[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types
[2]
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/types/#user-defined-data-types
On Tue, 5 Apr 2022 at 15:41, Anitha Thankappan <
anitha.thankappan@quantiphi.com> wrote:
> Hi,
>
> I am doing a developed a Fink connector for GCP BigQuery.
> When we are reading BIGNUMERIC data from BigQuery table, we didn't find
> any matching data types in Flink. Also failed to implement user defined
> data type for BIGNUMERIC.
>
> Please let me know if there is any way to handle this.
>
> Thanks in Advance,
> Anitha Thankappan
>
> *This message contains information that may be privileged or confidential
> and is the property of the Quantiphi Inc and/or its affiliates**. It is
> intended only for the person to whom it is addressed. **If you are not
> the intended recipient, any review, dissemination, distribution, copying,
> storage or other use of all or any portion of this message is strictly
> prohibited. If you received this message in error, please immediately
> notify the sender by reply e-mail and delete this message in its *
> *entirety*
>