You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/20 08:13:02 UTC

[GitHub] [airflow] highfly22 opened a new pull request #15448: Add quoteChar property in the Hive table creation hql

highfly22 opened a new pull request #15448:
URL: https://github.com/apache/airflow/pull/15448


   This patch try to solve the issue that csv fields contains delimiter character. For example,
   
   ```
   ,id,json,order_id
   0,"[{'id': 192, 'name': 'foo'}]",20
   
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-842175573


   [The Workflow run](https://github.com/apache/airflow/actions/runs/849193584) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] RosterIn commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
RosterIn commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-828208678


   there is a pylint issue
   `airflow/providers/apache/hive/hooks/hive.py:377:4: R0913: Too many arguments (11/10) (too-many-arguments)`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-887109418


   This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-842175573


   [The Workflow run](https://github.com/apache/airflow/actions/runs/849193584) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-841083885


   [The Workflow run](https://github.com/apache/airflow/actions/runs/841403798) is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] highfly22 closed pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
highfly22 closed pull request #15448:
URL: https://github.com/apache/airflow/pull/15448


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-841224181


   [The Workflow run](https://github.com/apache/airflow/actions/runs/839587614) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] highfly22 closed pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
highfly22 closed pull request #15448:
URL: https://github.com/apache/airflow/pull/15448


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#issuecomment-823092862


   [The Workflow run](https://github.com/apache/airflow/actions/runs/766429741) is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] highfly22 closed pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
highfly22 closed pull request #15448:
URL: https://github.com/apache/airflow/pull/15448


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] closed pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #15448:
URL: https://github.com/apache/airflow/pull/15448


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb commented on a change in pull request #15448: Add quoteChar property in the Hive table creation hql

Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #15448:
URL: https://github.com/apache/airflow/pull/15448#discussion_r649305935



##########
File path: airflow/providers/apache/hive/hooks/hive.py
##########
@@ -377,11 +377,13 @@ def _infer_field_types_from_df(df: pandas.DataFrame) -> Dict[Any, Any]:
                     filepath=f.name, table=table, delimiter=delimiter, field_dict=field_dict, **kwargs
                 )
 
+    # pylint: disable=too-many-arguments
     def load_file(
         self,
         filepath: str,
         table: str,
         delimiter: str = ",",
+        quotechar: str = '\\"',

Review comment:
       This should just be
   
   ```suggestion
           quotechar: str = '"',
   ```
   
   and then any escaping necessary should be handled in the SQL generation.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org