Get SQL statements from QueryExecutionListener

[Thắng Nguyễn Đình <th...@gmail.com>]

Hi guys,

At the moment, I want to integrate Spark SQL execution to Apache Atlas to
show the lineage:

`source_table` --SQL statement--`target_table`

I have tried SAC [https://github.com/hortonworks-spark/spark-atlas-connector]
but it does not support Spark 3.x. And SAC lacks SQL statements for Atlas
processes, only have Plans.

My approach is create a Listener with QueryExecutionListener and I have two
questions
- How can we get SQL statements from a QueryExecution object?
- If we can not, whether we can track SQL statements
by QueryPlanningTracker?

Thanks for your help!


Best wish

------------------------------------------------------
Nguyễn Đình Thắng