You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/09/01 01:11:02 UTC
[GitHub] [dolphinscheduler] cxin-william opened a new issue #6076: Can we join greenplum？
cxin-william opened a new issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076
Since writing greenplum using datax is very slow with a PostgresQL writer, you can write GP using GreenplumWriter, which is about five times faster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-994158392
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhuangchong commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
zhuangchong commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-911382852
@cxin-william
Good idea, can you submit a PR to contribute to this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] cxin-william commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
cxin-william commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-912371525
I think just add greenplum to the data source,right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] cxin-william commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
cxin-william commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-912388487
Our datax is installed under /opt/datax, which already has greenplum Writer and greenplum Reader
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005332514
Close this issue due to https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005317427. Private advice, since datax seem not to upgrade for a long time, you could possibility use https://github.com/apache/incubator-seatunnel instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] gaogen123 commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
gaogen123 commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005410929
ok, could you will add support for greenplum? This very import for most of greenplum users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhuangchong commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
zhuangchong commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-912379801
> I think just add greenplum to the data source,right?
1. Need to add Greenplum Writer and Reader to the datax task class
2. Data source supports Greenplum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] gaogen123 commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
gaogen123 commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1004771240
You can refer to this link
https://github.com/HashDataInc/DataX/blob/master/gpdbwriter/src/main/java/cn/hashdata/datax/plugin/writer/gpdbwriter/CopyProcessor.java 。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-999262030
Hi, @cxin-william are you interested in this issue? If so I could assigned to you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-909779562
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005428611
> I mean add greenplum datasource ,not datax
@gaogen123 Then you could create new issue to about support greenplum datasource, cause this issue is about
> Since writing greenplum using datax is very slow with a PostgresQL writer, you can write GP using GreenplumWriter, which is about five times faster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie closed issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
zhongjiajie closed issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] closed issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-999234826
Remove stale
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-999180732
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] gaogen123 commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
gaogen123 commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005422900
I mean add greenplum datasource ,not datax
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005317427
> You can refer to this link https://github.com/HashDataInc/DataX/blob/master/gpdbwriter/src/main/java/cn/hashdata/datax/plugin/writer/gpdbwriter/CopyProcessor.java 。
I seem this link is for datax but not for any fork of DolphinScheduler, if here we only need to add datax writer, you possibility just need to submit PR to datax, not DolphinScheduler @gaogen123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6076: Can we join greenplum？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-909779562
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] gaogen123 commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
gaogen123 commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005410529
ok, could you will add support for greenplum? This very import for most of greenplum users. Although Postgresql also can connect. However, we ran into a lot of problems when running Greenplum procedures and dataX tasks concurrently with the postgresql driver in DS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on issue #6076: [Feature] Add support of greenplum
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on issue #6076:
URL: https://github.com/apache/dolphinscheduler/issues/6076#issuecomment-1005415231
>
Hi @gaogen123 , we should not and can not add grenplum support is DS, it is not dolphinscheduler scope, it belong to datax, we can not change datax source code, I think it better create an issue for datax, but not for dolphinscheduler
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org