[GitHub] [iceberg] brandonstanleyappfolio commented on issue #7627: Failing to upsert/delete records in Apache Iceberg Tables using `MERGE INTO` via Spark Structured Streaming (PySpark)

["brandonstanleyappfolio (via GitHub)" <gi...@apache.org>]

brandonstanleyappfolio commented on issue #7627:
URL: https://github.com/apache/iceberg/issues/7627#issuecomment-1625718566

   This issue was resolved. The root cause of this problem was when the PySpark shell is not terminated correctly (i.e. without explicitly calling `exit()`). Here are the steps to re-create the problem: 
   1. Start PySpark shell
   2. Execute the structured streaming code above
   3. Close terminal while PySpark shell is still running (without invoking the `exit()` function)
   4. Re-open PySpark shell
   5. Execute the structured streaming code above
    
   I was able to successfully execute the [original code](https://github.com/apache/iceberg/issues/7627#issue-1713029907) after re-creating my Spark environment entirely (i.e. re-creating the Kubernetes pod that I was using to test my code).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org