You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/10/18 03:17:54 UTC
[GitHub] [dolphinscheduler] yyh2954360585 opened a new issue, #12409: When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
yyh2954360585 opened a new issue, #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
I found that after the server netstat spark port The worker server does not disconnect from the Sparkhrift after the connection is established I think the SQL (spark data source) task node in each workflow will remain connected after execution.Restores to normal after the DS service is stopped.
spark version:3.2.1
hive version:3.1.2
### What you expected to happen
The worker server did not provide clear error information.
tail worker log file:
netstat spark thrift server pord status:
The status and connect open num of the spark thrift port after the DS service is stopped:
### How to reproduce
1.create spark datasource
2.Define workflow and create spark data source query statement in workflow
3.run workflow
4.Connection status of the port after execution
### Anything else
_No response_
### Version
3.0.x
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] songjianet commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
songjianet commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1282437813
Please confirm if the issue exists in 31x and dev.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] yyh2954360585 commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
yyh2954360585 commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1282456335
> Please confirm if the issue exists in 31x and dev.
OK, I'll try it tomorrow. thank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] closed issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
URL: https://github.com/apache/dolphinscheduler/issues/12409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] yyh2954360585 commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
yyh2954360585 commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1326933397
> > The problem still exists after I upgrade DS to version 3.1x
>
> @yyh2954360585 can you fix it?
will try to solve this problem when I have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1418313753
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] yyh2954360585 commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
yyh2954360585 commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1318101880
The problem still exists after I upgrade DS to version 3.1x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] davidzollo commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
davidzollo commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1326452368
> The problem still exists after I upgrade DS to version 3.1x
@yyh2954360585 can you fix it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12409: When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1281759999
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wendongdi commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
wendongdi commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1369451409
has this bug been fixed ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1427177994
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] yyh2954360585 commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
yyh2954360585 commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1326929306
>
I will try to solve this problem when I have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] yyh2954360585 commented on issue #12409: [Bug][Worker Server ] When the DS uses the data source to connect to the Spark Thrift server, the connection channel remains connected and will not be closed
Posted by GitBox <gi...@apache.org>.
yyh2954360585 commented on issue #12409:
URL: https://github.com/apache/dolphinscheduler/issues/12409#issuecomment-1326933618
> > The problem still exists after I upgrade DS to version 3.1x
>
> @yyh2954360585 can you fix it?
I will try to solve this problem when I have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org