You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/05/07 06:11:39 UTC

[GitHub] [incubator-doris] yaowanli opened a new issue #5760: Flink-doris

yaowanli opened a new issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] yaowanli commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
yaowanli commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-835998770


   Thank you for your reply.
   I have seen sequence column function which is useful ,I think it can solve the second problem which I mention. but it is still in the master branch.I looking forward to the stable version of it so that I can update the version when I use branch-0.12 now. 
   Howerver,the first problem, data uodate late, may need to be considered.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] sniperxing commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
sniperxing commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-871556007


   sequence在flink doris connecter中如何设置呢?我看文档中没有对应的参数


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] JNSimba commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
JNSimba commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-877568312


   > > > > sequence在flink doris connecter中如何设置呢?我看文档中没有对应的参数
   > > > 
   > > > 
   > > > Set when the table is created
   > > > http://doris.apache.org/master/zh-CN/administrator-guide/load-data/sequence-column-manual.html#%E4%BD%BF%E7%94%A8%E7%A4%BA%E4%BE%8B
   > > 
   > > 
   > > 建表的时候只能设置sequence的type
   > > 而flink doris connecter中使用的是stream load,stram load需要在header中设置sequence coloum,但是flink doris connecter中没有提供该参数呀
   > 
   > well, currently, Flink connector does not support this parameter
   
   #6199 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] JNSimba commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
JNSimba commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-872684647


   > sequence在flink doris connecter中如何设置呢?我看文档中没有对应的参数
   
   Set when the table is created 
   http://doris.apache.org/master/zh-CN/administrator-guide/load-data/sequence-column-manual.html#%E4%BD%BF%E7%94%A8%E7%A4%BA%E4%BE%8B


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] JNSimba commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
JNSimba commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-835644608


   1.In the document:[sequence column](http://doris.apache.org/master/zh-CN/administrator-guide/load-data/sequence-column-manual.html#%E4%BD%BF%E7%94%A8%E7%A4%BA%E4%BE%8B)  function compares time type or incremental type fields. If there is no time type field in the table, how to ensure that the data is out of order?
   2.For the uniq model, I tested it. Streamload is inserted according to the sequence of the data. In what scenarios will there be disorder?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] morningman commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
morningman commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-835624170


   这种有序需要用到 doris的 sequence column 功能,单独的 unique无法保证


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] sniperxing commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
sniperxing commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-874751235


   
   
   
   > > sequence在flink doris connecter中如何设置呢?我看文档中没有对应的参数
   > 
   > Set when the table is created
   > http://doris.apache.org/master/zh-CN/administrator-guide/load-data/sequence-column-manual.html#%E4%BD%BF%E7%94%A8%E7%A4%BA%E4%BE%8B
   
   建表的时候只能设置sequence的type
   而flink doris connecter中使用的是stream load,stram load需要在header中设置sequence coloum,但是flink doris connecter中没有提供该参数呀


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] JNSimba commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
JNSimba commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-835621453


   1.thank you so much for your suggestion,This piece of follow-up needs to be optimized
   2.Since write to doris is based on streamload, if your table is a UNIQUE model, it will automatically update based on the sequence of data


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] JNSimba removed a comment on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
JNSimba removed a comment on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-835644608


   1.In the document:[sequence column](http://doris.apache.org/master/zh-CN/administrator-guide/load-data/sequence-column-manual.html#%E4%BD%BF%E7%94%A8%E7%A4%BA%E4%BE%8B)  function compares time type or incremental type fields. If there is no time type field in the table, how to ensure that the data is out of order?
   2.For the uniq model, I tested it. Streamload is inserted according to the sequence of the data. In what scenarios will there be disorder?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] JNSimba commented on issue #5760: Flink-doris数据更新和数据重复

Posted by GitBox <gi...@apache.org>.
JNSimba commented on issue #5760:
URL: https://github.com/apache/incubator-doris/issues/5760#issuecomment-876082090


   > > > sequence在flink doris connecter中如何设置呢?我看文档中没有对应的参数
   > > 
   > > 
   > > Set when the table is created
   > > http://doris.apache.org/master/zh-CN/administrator-guide/load-data/sequence-column-manual.html#%E4%BD%BF%E7%94%A8%E7%A4%BA%E4%BE%8B
   > 
   > 建表的时候只能设置sequence的type
   > 而flink doris connecter中使用的是stream load,stram load需要在header中设置sequence coloum,但是flink doris connecter中没有提供该参数呀
   
   well, currently, Flink connector does not support this parameter
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org