You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/08/04 03:22:33 UTC
[GitHub] [hudi] Ares-W opened a new issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Ares-W opened a new issue #1908:
URL: https://github.com/apache/hudi/issues/1908
There are too many close_wait sockets in 50010 when I use SparkStreaming to save kafka data to Hudi. I use dataFrame.write.format("org.apache.hudi").mode().save() method to save data to hudi. But when one batch data write to hudi,there remains some close_wait sockets on 50010 port and never close,With the program running,there are more and more close_wait socket on 50010 and throw “Too mang open files” Exception. How to close the close_wait socket?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Ares-W commented on issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Posted by GitBox <gi...@apache.org>.
Ares-W commented on issue #1908:
URL: https://github.com/apache/hudi/issues/1908#issuecomment-734622537
> yikes. did you downgrade manually. how did you end up solving this? it will be helpful for others if you can share
just changed the parquet version from 1.9.0 to 1.8.3
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1908:
URL: https://github.com/apache/hudi/issues/1908#issuecomment-675755135
yikes. did you downgrade manually. how did you end up solving this? it will be helpful for others if you can share
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] kimmazhenxin commented on issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Posted by GitBox <gi...@apache.org>.
kimmazhenxin commented on issue #1908:
URL: https://github.com/apache/hudi/issues/1908#issuecomment-688614721
> parquet-1.9.0 cause this problem
> https://issues.apache.org/jira/browse/PARQUET-783
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Ares-W closed issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Posted by GitBox <gi...@apache.org>.
Ares-W closed issue #1908:
URL: https://github.com/apache/hudi/issues/1908
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] Ares-W commented on issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Posted by GitBox <gi...@apache.org>.
Ares-W commented on issue #1908:
URL: https://github.com/apache/hudi/issues/1908#issuecomment-668410651
parquet-1.9.0 cause this problem
https://issues.apache.org/jira/browse/PARQUET-783
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] kimmazhenxin removed a comment on issue #1908: [SUPPORT]There are too many close_wait sockets on 50010 port when I use SparkStreaming to save kafka data to Hudi.
Posted by GitBox <gi...@apache.org>.
kimmazhenxin removed a comment on issue #1908:
URL: https://github.com/apache/hudi/issues/1908#issuecomment-688614721
> parquet-1.9.0 cause this problem
> https://issues.apache.org/jira/browse/PARQUET-783
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org