You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/11/21 07:59:21 UTC

[GitHub] [incubator-uniffle] xianjingfeng opened a new issue, #346: [Bug] `indexWriter` don't close if exception thrown when close `dataWriter ` in `HdfsShuffleWriteHandler`

xianjingfeng opened a new issue, #346:
URL: https://github.com/apache/incubator-uniffle/issues/346

   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   
   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Describe the bug
   
   I found there are many threads in shuffle server as follows:
   
   ```
   "DataStreamer for file /uniffle/data/application_1662640300316_231973311668726143398/4/138-138/10.xx.xx.xx-59999_0.index block BP-150738345-10.116.103.12-1553601482353:blk_4236218692_3237686719" #1062624
   daemon prio=5 os_prio=0 tid=0x00007fb184030000 nid=0x1b323 in Object.wait() [0x00007faf48d0f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:417)
   - locked <0x00007fba52000f90> (a java.util.LinkedList)
   
   Locked ownable synchronizers:
   - None
   ```
   
   And all `DataStreamer` is for index file.
   
   ### Affects Version(s)
   
   master
   
   ### Uniffle Server Log Output
   
   _No response_
   
   ### Uniffle Engine Log Output
   
   _No response_
   
   ### Uniffle Server Configurations
   
   _No response_
   
   ### Uniffle Engine Configurations
   
   _No response_
   
   ### Additional context
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi closed issue #346: [Bug] `indexWriter` don't close if exception thrown when close `dataWriter ` in `HdfsShuffleWriteHandler`

Posted by GitBox <gi...@apache.org>.
jerqi closed issue #346: [Bug] `indexWriter` don't close if exception thrown when close `dataWriter ` in `HdfsShuffleWriteHandler`
URL: https://github.com/apache/incubator-uniffle/issues/346


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #346: [Bug] `indexWriter` don't close if exception thrown when close `dataWriter ` in `HdfsShuffleWriteHandler`

Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #346:
URL: https://github.com/apache/incubator-uniffle/issues/346#issuecomment-1321782730

   Good catch!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org