Fwd: Issue in Null handling in python snowflakeIO

[Abhinav Jha <ab...@datametica.com>]

---------- Forwarded message ---------
From: Abhinav Jha <ab...@datametica.com>
Date: Wed, May 4, 2022 at 7:59 PM
Subject: [Question] Issue in Null handling in python snowflakeIO
To: <us...@beam.apache.org>


Hi,
I am trying to insert data from snowflake to bigquery using python
*ReadFromSnowflake
*beam.
Everything is working fine except when there is a *null value* in the
snowflake column. If there is a null, data read by snowflake comes as *b'N'*
and hence its a wrong data.
I  am attaching the snowflake data along with the data that I have printed
in the csv_mapper of ReadFromSnowflake.
Any suggestions or help regarding this will be helpful.
Also attaching my csv mapper function.

def getJsonRow(self, mylist):
    schemaList = self.schema.get('fields')
    headerList = []
    jsonDict = {}
    for ele in schemaList:
        headerList.append(ele.get('name'))
    cnt = 0
    for ele in mylist:
        jsonDict.update({headerList[cnt]: ele.decode()})
        cnt += 1
    source_name =
f"{self.prop_dict['source_db']['schema']}.{self.prop_dict['source_db']['database']}.{self.prop_dict['source_db']['snowflake_table']}"
    jsonDict.update({"source_name": source_name})
    return dict(jsonDict)

def csv_mapper(self, strings_array):
    return self.getJsonRow(strings_array)


Thanks
Abhinav Jha