Working out the optimizer matrix in Spark

[Mich Talebzadeh <mi...@cloudtechnologypartners.co.uk>]

 

Hi, 

I would like to know if there are commands available with spark to allow
to see all active processes plus the details of each process. 

FYI I am aware of cluster information in Spark GUI on port 4040. What I
am specifically looking is details from the optimiser itself, the
physical and logical IOs and locks on tables 

For example if the locks held much like Hive command that show locks 

Lock ID Database Table Partition State Type Transaction ID Last Hearbeat
Acquired At User Hostname
6077 oraclehadoop sales_staging NULL ACQUIRED SHARED_READ NULL
1455573580652 1455573578990 hduser rhes564 

Thanks, 
-- 

Dr Mich Talebzadeh

LinkedIn
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

http://talebzadehmich.wordpress.com

NOTE: The information in this email is proprietary and confidential.
This message is for the designated recipient only, if you are not the
intended recipient, you should destroy it immediately. Any information
in this message shall not be understood as given or endorsed by Cloud
Technology Partners Ltd, its subsidiaries or their employees, unless
expressly so stated. It is the responsibility of the recipient to ensure
that this email is virus free, therefore neither Cloud Technology
partners Ltd, its subsidiaries nor their employees accept any
responsibility.