[jira] [Closed] (ARROW-5799) [Python] Fail to write nested data to Parquet via BigQuery API

["Wes McKinney (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney closed ARROW-5799.
-------------------------------
    Resolution: Duplicate

> [Python] Fail to write nested data to Parquet via BigQuery API
> --------------------------------------------------------------
>
>                 Key: ARROW-5799
>                 URL: https://issues.apache.org/jira/browse/ARROW-5799
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.13.0
>         Environment: Python 3.6
>            Reporter: David Draper
>            Priority: Major
>
> I keep gettting the error in the title. any ideas on how to fix this issue? 
> for company, credentials in loginCredentials.items():
>  password = credentials["Password"]
>  username = credentials["Username"]
>  Academy = company
>  Phase = credentials["Phase"]
>  values = \{"grant_type": "password","username": username, "password": password}
>  data = urlencode(values).encode()
>  session = requests.Session()
>  session.headers = {
>  'Content-Type': 'application/x-www-form-urlencoded'
>  }
>  response_body = session.post(TOKEN_API_URL, data=data)
>  access_token = json.loads(response_body.text)["access_token"]
>  #print(Academy +" "+ str(response_body.status_code)+" "+response_body.reason)
>  session.headers = {
>  'Authorization': 'Bearer {}'.format(access_token)
>  }
>  #print(username + access_token)
>  learner_responses = session.get(LEARNER_API_URL)
>  learner_exclusions = session.get(LEARNER_EXCLUSIONS_URL)
>  #print(Academy + " "+ str(learner_responses.status_code) +" "+ learner_responses.reason)
>  if learner_responses.status_code == 200:
>  response = json.loads(learner_responses.text)
>  learners = pd.DataFrame(response)
>  learners['Establishment_Name'] = Academy
>  learners['Establishment_Phase'] = Phase
>  entries.append(learners)
>  else:
>  continue
>  
> appended_data = pd.concat(entries, ignore_index=True)
>  
> from google.cloud import bigquery
> project = 'aet-data-lake'
> client = bigquery.Client(credentials=credentials, project=project)
> dataset_ref = client.dataset('RAW')
> table_ref = dataset_ref.table('Learners_AET')
> job_config = bigquery.LoadJobConfig()
> job_config.autodetect = True
> client.load_table_from_dataframe(appended_data, table_ref,job_config=job_config).result()



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)