[GitHub] [spark] KevinAppelBofa commented on pull request #34709: [WIP][SPARK-37259] Add option to unwrap query to support CTE for MSSQL JDBC driver

[GitBox <gi...@apache.org>]

KevinAppelBofa commented on pull request #34709:
URL: https://github.com/apache/spark/pull/34709#issuecomment-981732735


   @akhalymon-cv thank you for working on the patch, I was able to test this and both the test queries and the actual temp and CTE queries are working.  the item in the example you show using query however if I do this, then I get an error and in instead the query must be passed via the dbtable option
   For the temp table query this is having a similar run time as the patches from @peter-toth has created; the temp table was easy to split but I haven't been able to split the CTE yet for a comparison.
   For these particular queries, they are so complicated currently that it would be difficult to find a way to split them up for parallel; currently we are using either predicates or numpartitions when we are doing the parallel of the jdbc.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org