You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@hop.apache.org by "Galewsky, Andrew" <an...@galewsky.net> on 2023/05/22 18:03:51 UTC
Hop and Casts for Postgres
I have been fighting this since I started using hop. For example I have
a database field that is boolean but the incoming field has some dirty data
and is not able to be set as a bool in its temp table. It has to be a
varchar. As part of the pipeline the data gets cleaned up and is now
truly boolean. As far as I can tell now there is no way to cast it as bool
before writing to the output table. Am I dense or missing something?.
-Andrew Galewsky
Re: Hop and Casts for Postgres
Posted by ha...@gmail.com.
Hi Andrew,
You can use the select values transform and in there use the metadata tab to change the datatype of a field. This can be used to change dates to strings or int/strings to boolean and so on [1].
Hope this helps,
Kr,
Hans
[1] https://hop.apache.org//manual/latest/pipeline/transforms/selectvalues.html#_meta_data_tab
On 22 May 2023 at 21:20 +0200, Galewsky, Andrew <an...@galewsky.net>, wrote:
> As far as I know there is nothing specific about that in the connection for postgresql....
>
> -Andrew
>
> > On Mon, May 22, 2023 at 1:39 PM Matt Casters <ma...@leanwithdata.com> wrote:
> > > Did you enable boolean support in the relational database connection?
> > >
> > >
> > >
> > > > Op ma 22 mei 2023 20:04 schreef Galewsky, Andrew <an...@galewsky.net>:
> > > > > I have been fighting this since I started using hop. For example I have a database field that is boolean but the incoming field has some dirty data and is not able to be set as a bool in its temp table. It has to be a varchar. As part of the pipeline the data gets cleaned up and is now truly boolean. As far as I can tell now there is no way to cast it as bool before writing to the output table. Am I dense or missing something?.
> > > > >
> > > > > -Andrew Galewsky
Re: Hop and Casts for Postgres
Posted by "Galewsky, Andrew" <an...@galewsky.net>.
Matt -
You are correct and I was wrong - I had never seen that. Set it and
retesting...
-Andrew
On Mon, May 22, 2023 at 3:27 PM Matt Casters <ma...@leanwithdata.com>
wrote:
> I'm asking since the option is not enabled by default in the Hop Metadata
> element. Boolean is not a standard SQL datatype so Hop wants to know if
> you want to use it.
>
>
> Op ma 22 mei 2023 21:20 schreef Galewsky, Andrew <an...@galewsky.net>:
>
>> As far as I know there is nothing specific about that in the connection
>> for postgresql....
>>
>> -Andrew
>>
>> On Mon, May 22, 2023 at 1:39 PM Matt Casters <
>> matt.casters@leanwithdata.com> wrote:
>>
>>> Did you enable boolean support in the relational database connection?
>>>
>>>
>>>
>>> Op ma 22 mei 2023 20:04 schreef Galewsky, Andrew <an...@galewsky.net>:
>>>
>>>> I have been fighting this since I started using hop. For example I
>>>> have a database field that is boolean but the incoming field has some dirty
>>>> data and is not able to be set as a bool in its temp table. It has to be a
>>>> varchar. As part of the pipeline the data gets cleaned up and is now
>>>> truly boolean. As far as I can tell now there is no way to cast it as bool
>>>> before writing to the output table. Am I dense or missing something?.
>>>>
>>>> -Andrew Galewsky
>>>>
>>>
Re: Hop and Casts for Postgres
Posted by Matt Casters <ma...@leanwithdata.com>.
I'm asking since the option is not enabled by default in the Hop Metadata
element. Boolean is not a standard SQL datatype so Hop wants to know if
you want to use it.
Op ma 22 mei 2023 21:20 schreef Galewsky, Andrew <an...@galewsky.net>:
> As far as I know there is nothing specific about that in the connection
> for postgresql....
>
> -Andrew
>
> On Mon, May 22, 2023 at 1:39 PM Matt Casters <
> matt.casters@leanwithdata.com> wrote:
>
>> Did you enable boolean support in the relational database connection?
>>
>>
>>
>> Op ma 22 mei 2023 20:04 schreef Galewsky, Andrew <an...@galewsky.net>:
>>
>>> I have been fighting this since I started using hop. For example I
>>> have a database field that is boolean but the incoming field has some dirty
>>> data and is not able to be set as a bool in its temp table. It has to be a
>>> varchar. As part of the pipeline the data gets cleaned up and is now
>>> truly boolean. As far as I can tell now there is no way to cast it as bool
>>> before writing to the output table. Am I dense or missing something?.
>>>
>>> -Andrew Galewsky
>>>
>>
Re: Hop and Casts for Postgres
Posted by "Galewsky, Andrew" <an...@galewsky.net>.
As far as I know there is nothing specific about that in the connection for
postgresql....
-Andrew
On Mon, May 22, 2023 at 1:39 PM Matt Casters <ma...@leanwithdata.com>
wrote:
> Did you enable boolean support in the relational database connection?
>
>
>
> Op ma 22 mei 2023 20:04 schreef Galewsky, Andrew <an...@galewsky.net>:
>
>> I have been fighting this since I started using hop. For example I
>> have a database field that is boolean but the incoming field has some dirty
>> data and is not able to be set as a bool in its temp table. It has to be a
>> varchar. As part of the pipeline the data gets cleaned up and is now
>> truly boolean. As far as I can tell now there is no way to cast it as bool
>> before writing to the output table. Am I dense or missing something?.
>>
>> -Andrew Galewsky
>>
>
Re: Hop and Casts for Postgres
Posted by Matt Casters <ma...@leanwithdata.com>.
Did you enable boolean support in the relational database connection?
Op ma 22 mei 2023 20:04 schreef Galewsky, Andrew <an...@galewsky.net>:
> I have been fighting this since I started using hop. For example I have
> a database field that is boolean but the incoming field has some dirty data
> and is not able to be set as a bool in its temp table. It has to be a
> varchar. As part of the pipeline the data gets cleaned up and is now
> truly boolean. As far as I can tell now there is no way to cast it as bool
> before writing to the output table. Am I dense or missing something?.
>
> -Andrew Galewsky
>