You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Joseph PENG <jo...@gmail.com> on 2017/10/30 00:04:27 UTC

[Spark Debug] Search vars of interest

I am debug spark (itself not its application) by running `sbt  default test
suite`. I have successfully set up everything, and IntelliJ debugger have
caught every variables. However, I can not find the variables I need,
because there are so many variables.

For example, I want to find the input data frame, the transformed data
frame, and the output data frame. Where should I look at?

What I tried:
1. use search function in IntelliJ to search variable name.
2. try to understand how scaliest works.

Can someone point out a direction?

Screenshot of debugger
https://www.dropbox.com/s/29p98ow5sp2h7u4/Screenshot%202017-10-29%2019.02.59.png?dl=0

Thanks!