You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "hantiantian (JIRA)" <ji...@apache.org> on 2019/01/25 08:52:00 UTC
[jira] [Created] (SPARK-26726) The amount of memory used by the
broadcast variable is not synchronized to the UI display
hantiantian created SPARK-26726:
-----------------------------------
Summary: The amount of memory used by the broadcast variable is not synchronized to the UI display
Key: SPARK-26726
URL: https://issues.apache.org/jira/browse/SPARK-26726
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.4.0
Reporter: hantiantian
The amount of memory used by the broadcast variable is not synchronized to the UI display，
spark-sql> select /*+ broadcast(a)*/ a.id,b.id from a join b on a.id = b.id;
View the app's driver log：
2019-01-25 16:45:23,726 INFO org.apache.spark.storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.43.xx.xx:33907 (size: 6.6 KB, free: 2.5 GB)
2019-01-25 16:45:23,727 INFO org.apache.spark.storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 10.43.xx.xx:38399 (size: 6.6 KB, free: 2.5 GB)
2019-01-25 16:45:23,745 INFO org.apache.spark.storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.43.xx.xx:33907 (size: 32.1 KB, free: 2.5 GB)
2019-01-25 16:45:23,749 INFO org.apache.spark.storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 10.43.xx.xx:38399 (size: 32.1 KB, free: 2.5 GB)
2019-01-25 16:45:23,838 INFO org.apache.spark.storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.43.xx.xx:38399 (size: 147.0 B, free: 2.5 GB)
2019-01-25 16:45:23,840 INFO org.apache.spark.storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 10.43.xx.xx:33907 (size: 147.0 B, free: 2.5 GB)
Web UI does not have the use of memory，
!image-2019-01-25-16-48-28-435.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org