You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Sai Hemanth Gantasala (Jira)" <ji...@apache.org> on 2024/01/19 19:31:00 UTC

[jira] [Commented] (IMPALA-8518) Investigate support for insert events from SparkSQL

    [ https://issues.apache.org/jira/browse/IMPALA-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17808788#comment-17808788 ] 

Sai Hemanth Gantasala commented on IMPALA-8518:
-----------------------------------------------

This issue is fixed via HIVE-27173. The event processor in Impala can now consume DML events generated by Spark. 

> Investigate support for insert events from SparkSQL
> ---------------------------------------------------
>
>                 Key: IMPALA-8518
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8518
>             Project: IMPALA
>          Issue Type: Task
>          Components: Frontend
>    Affects Versions: Impala 3.2.0
>            Reporter: Bharath Krishna
>            Assignee: Sai Hemanth Gantasala
>            Priority: Major
>
> Hive can issue insert events when data is inserted into tables and partitions. These events are used to update Impala's catalog to refresh the file metadata information for such tables and partitions. We should investigate if we can do something similar for Spark so that ETL jobs run using spark can also be supported.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org