You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "banmoy (via GitHub)" <gi...@apache.org> on 2023/10/11 11:18:13 UTC
[I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
banmoy opened a new issue, #5404:
URL: https://github.com/apache/kyuubi/issues/5404
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before creating
- [X] I have searched in the [task list](https://github.com/orgs/apache/projects/296) and found no similar tasks.
### Mentor
- [ ] I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task.
### Skill requirements
* Basic knowledge of Scala programming language
* Basic knowledge of JDBC concepts
* Basic knowledge of Docker and [Testcontianers](https://java.testcontainers.org/)
### Background and Goals
Currently, Kyuubi supports JDBC engines with limited dialects, and we can extend the dialects to support [StarRocks](https://www.starrocks.io/), which is an open-source and high-performance analytical database.
Reference https://www.bilibili.com/video/BV1UT411m7LR
### Implementation steps
* Implement a StarRocks dialect in the JDBC engine
* Leverage the Testcontainers to add integration tests
* Update document correspondingly
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1778681326
The latest stable released version of StarRocks is 3.1.x, which is reasonable to be tested with.
And feel free to raise the PR, even not unfinished. Just remember to mark the PR in the draft status.
By running CI tests, the failing tests and fixes bring it to more mutual.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "zhaomin1423 (via GitHub)" <gi...@apache.org>.
zhaomin1423 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759643781
> Let's assign this task to both @phionex0571 @banmoy first. Please raise the subtasks for this task. Small subtasks and improvements are always welcomed. @phionex0571 please feel free to ask @banmoy about details of StartRocks, since he is from StarRocks team and has professionalism in Spark/Flink connector as well.
>
> Also cc @zhaomin1423, the original contributor of the JDBC engine. They may need some guidance for this task.
>
> Thanks to all.
Thanks you all, Starrocks is excellent ans used widely, look forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "phionex0571 (via GitHub)" <gi...@apache.org>.
phionex0571 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759123514
@pan3793 Hello, I have strong interests on it and during my daily work, Apache Kyuubi and StarRocks are two most useful tools. I think it can be finished in 7 days! please aisgn it to me~thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "banmoy (via GitHub)" <gi...@apache.org>.
banmoy commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757700133
Thanks a lot :smiley:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757914077
Thanks for raising this task. We could split the task into several smaller subtasks to accomplish.
You could be assigned to this task first, and submit the relevant PR to it.
Will try to help if there's somewhere you need a hand or review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759618515
Let's assign this task to both @phionex0571 @banmoy first.
Please raise the subtasks for this task. Small subtasks and improvements are always welcomed.
@phionex0571 please feel free to ask @banmoy about details of StartRocks, since he is from StarRocks team and has professionalism in Spark/Flink connector as well.
Also cc @zhaomin1423, the original contributor of the JDBC engine. They may need some guidance for this task.
Thanks to all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757459348
Hello @banmoy,
Thanks for finding the time to report the issue!
We really appreciate the community's efforts to improve Apache Kyuubi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757703287
@banmoy would you like to complete it by yourself, or leave it to other contributors?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "banmoy (via GitHub)" <gi...@apache.org>.
banmoy commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759592284
I'm not sure I could start the work ASAP. I think the task can be assigned to other contributors if they have big interests in it. I'm also glad to provide the knowledge of StarRocks if needed in discussion, review or test. WDYT? @bowenliang123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 closed issue #5404: [TASK][EASY] JDBC Engine supports StarRocks
URL: https://github.com/apache/kyuubi/issues/5404
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759235095
@phionex0571 Emm... you have been assigned 2 tasks, do you have sufficient free time to take the third and complete them in parallel?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "phionex0571 (via GitHub)" <gi...@apache.org>.
phionex0571 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1778674619
@bowenliang123 Hi, I've finished almost all the coding work and I have a question about StarRocks version in test cases. There're
several versions of StarRocks and which one should I choose to design unit tests, is it reasonable to use StarRocks 2.5.13? or 3.x?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1865846174
@banmoy StarRocks support of dialects/connection/tests in the JDBC engine has been introduced in PR 5882 (https://github.com/apache/kyuubi/pull/5882), if you may be concerned. Have check or validation on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "banmoy (via GitHub)" <gi...@apache.org>.
banmoy commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757492405
@pan3793 could this issue be added to the task list of Code Contribution Program?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757920005
Btw, consider using the official docker image from StarRocks for testing, which could be helpful and easy to maintain.
refer : https://docs.starrocks.io/en-us/latest/quick_start/deploy_with_docker
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "phionex0571 (via GitHub)" <gi...@apache.org>.
phionex0571 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759487333
> @phionex0571 Emm... you have been assigned 2 tasks, do you have sufficient free time to take the third and complete them in parallel?
@pan3793 Yeeep, in the previous year, I'd finished a StarRocks proxy in Apache Kyuubi 1.4, but that was not elegant and that was imitated like Apache Doris Engine implementation in Apache Kyuubi. And it's applied by 3 applications in production environment of my company. I hold strong belief that I'm able to finish this issue in time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1780423600
@banmoy It's not suggested to tested on multiple versions of StarRocks. As for other engines for Spark/Flink/Trino, we tested on different matrix combinations as Kyuubi use different version of API on these engines and has dedicated engine implementation and adaptations.
But as StarRocks introduced no major FE feature or breaking changes in protocol or dialect between 2.x and 3.x, and the JDBC engine uses the same version of Mysql Driver for connection, it's reasonable to test only on the latest stable version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "bowenliang123 (via GitHub)" <gi...@apache.org>.
bowenliang123 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1759302655
@banmoy Can I assign this task to you , and you can guide and cooperate with @phionex0571 ? In order to maximize the common knowledge from individuals and Professionalism from SR team.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "banmoy (via GitHub)" <gi...@apache.org>.
banmoy commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757715671
could first leave it to other contributors if they are interested in it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "banmoy (via GitHub)" <gi...@apache.org>.
banmoy commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1780402883
@phionex0571 Is it possible to make the unit tests run against the latest several StarRocks versions, such as 2.5, 3.0, and 3.1? So that we can tell users in the doc which version of StarRocks is supported, just like Spark/Flink/Trino? I think test cases should be same among different versions, but just use the specific docker image for each version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1757688177
@banmoy thanks, task was added to the list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [TASK][EASY] JDBC Engine supports StarRocks [kyuubi]
Posted by "banmoy (via GitHub)" <gi...@apache.org>.
banmoy commented on issue #5404:
URL: https://github.com/apache/kyuubi/issues/5404#issuecomment-1827772255
@phionex0571 Hi, is there any progress on this task?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org