You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Davies Liu (JIRA)" <ji...@apache.org> on 2015/08/11 18:55:46 UTC

[jira] [Updated] (SPARK-9827) Too many open files in TungstenExchange

     [ https://issues.apache.org/jira/browse/SPARK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Davies Liu updated SPARK-9827:
------------------------------
    Description: 
When run q19 on TPCDS (scale=5) dataset with 8G memory, it open 10k shuffle files, crash many things (even Chrome).

{code}
davies@localhost:~/work/spark$ jps
95385 Jps
95316 SparkSubmit
davies@localhost:~/work/spark$ lsof -p 95316 | wc -l
9827
davies@localhost:~/work/spark$ lsof -p 95316 | tail
java    95316 davies 9772r     REG                1,2      9522 97350739 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/2a/shuffle_0_112_0.data
java    95316 davies 9773r     REG                1,2      8449 97351388 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1a/shuffle_0_116_0.data
java    95316 davies 9774r     REG                1,2      8200 97351134 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/09/shuffle_0_113_0.data
java    95316 davies 9775r     REG                1,2      8057 97351941 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/05/shuffle_0_117_0.data
java    95316 davies 9776r     REG                1,2      8565 97351133 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/18/shuffle_0_114_0.data
java    95316 davies 9777r     REG                1,2      8185 97351942 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1c/shuffle_0_118_0.data
java    95316 davies 9778r     REG                1,2      8865 97351135 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/07/shuffle_0_115_0.data
java    95316 davies 9779r     REG                1,2      8255 97351987 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/3d/shuffle_0_119_0.data
java    95316 davies 9780r     REG                1,2      8449 97351388 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1a/shuffle_0_116_0.data
java    95316 davies 9781r     REG                1,2      9105 97352148 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/13/shuffle_0_120_0.data
{code}

  was:
When run q19 on TPCDS (scale=5) dataset, it open 10k shuffle files, crash many things (even Chrome).

{code}
davies@localhost:~/work/spark$ jps
95385 Jps
95316 SparkSubmit
davies@localhost:~/work/spark$ lsof -p 95316 | wc -l
9827
davies@localhost:~/work/spark$ lsof -p 95316 | tail
java    95316 davies 9772r     REG                1,2      9522 97350739 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/2a/shuffle_0_112_0.data
java    95316 davies 9773r     REG                1,2      8449 97351388 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1a/shuffle_0_116_0.data
java    95316 davies 9774r     REG                1,2      8200 97351134 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/09/shuffle_0_113_0.data
java    95316 davies 9775r     REG                1,2      8057 97351941 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/05/shuffle_0_117_0.data
java    95316 davies 9776r     REG                1,2      8565 97351133 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/18/shuffle_0_114_0.data
java    95316 davies 9777r     REG                1,2      8185 97351942 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1c/shuffle_0_118_0.data
java    95316 davies 9778r     REG                1,2      8865 97351135 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/07/shuffle_0_115_0.data
java    95316 davies 9779r     REG                1,2      8255 97351987 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/3d/shuffle_0_119_0.data
java    95316 davies 9780r     REG                1,2      8449 97351388 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1a/shuffle_0_116_0.data
java    95316 davies 9781r     REG                1,2      9105 97352148 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/13/shuffle_0_120_0.data
{code}


> Too many open files in TungstenExchange
> ---------------------------------------
>
>                 Key: SPARK-9827
>                 URL: https://issues.apache.org/jira/browse/SPARK-9827
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.5.0
>            Reporter: Davies Liu
>            Priority: Blocker
>
> When run q19 on TPCDS (scale=5) dataset with 8G memory, it open 10k shuffle files, crash many things (even Chrome).
> {code}
> davies@localhost:~/work/spark$ jps
> 95385 Jps
> 95316 SparkSubmit
> davies@localhost:~/work/spark$ lsof -p 95316 | wc -l
> 9827
> davies@localhost:~/work/spark$ lsof -p 95316 | tail
> java    95316 davies 9772r     REG                1,2      9522 97350739 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/2a/shuffle_0_112_0.data
> java    95316 davies 9773r     REG                1,2      8449 97351388 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1a/shuffle_0_116_0.data
> java    95316 davies 9774r     REG                1,2      8200 97351134 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/09/shuffle_0_113_0.data
> java    95316 davies 9775r     REG                1,2      8057 97351941 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/05/shuffle_0_117_0.data
> java    95316 davies 9776r     REG                1,2      8565 97351133 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/18/shuffle_0_114_0.data
> java    95316 davies 9777r     REG                1,2      8185 97351942 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1c/shuffle_0_118_0.data
> java    95316 davies 9778r     REG                1,2      8865 97351135 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/07/shuffle_0_115_0.data
> java    95316 davies 9779r     REG                1,2      8255 97351987 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/3d/shuffle_0_119_0.data
> java    95316 davies 9780r     REG                1,2      8449 97351388 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/1a/shuffle_0_116_0.data
> java    95316 davies 9781r     REG                1,2      9105 97352148 /private/var/folders/r1/j51v8t_x4bq6fqt43nymzddw0000gn/T/blockmgr-6dd1dd1b-735d-4eae-a7eb-72820e0a2e7b/13/shuffle_0_120_0.data
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org