You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/04/27 08:08:30 UTC
[GitHub] [incubator-doris] WingsGo opened a new issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
WingsGo opened a new issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402
**Is your feature request related to a problem? Please describe.**
As shown in the document, if user create a olap table with ssd, I think we should give them a error tip that the cluster has no avaliable ssd storage, instread of store the data in hdd disk.
```
Note that this parameter is just a "best effort" setting when storage_medium is specified. Even if no SSD storage media is set in the cluster, no error is reported and it is automatically stored in the available data directory
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo edited a comment on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
WingsGo edited a comment on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-621145565
Hi, @morningman I found that CheckDataSizeQuota will only check by logic but not check by real disk usage.I have some questions:
1. One solution is get real disk space avaliable group by storage medium, and check is there are avaliable space for specify storage medium when create or alter table/partition, But the solution has a problem that it may cannot constrain the existing partition, so, we should separate the FE's High Watermark into two aspect, for example hdd_storage_high_watermark_usage_percent && ssd_storage_high_watermark_usage_percent?
2. should we refacor the checkDataSizeQuota logic to limit the quota by real usage?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-619867481
Do you mean to reject the operation, or just return a tip to user but the creation still succeed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo edited a comment on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
WingsGo edited a comment on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-621145565
@morningman I found that CheckDataSizeQuota will only check by logic but not check by real disk usage.I have some questions:
1. One solution is get real disk space avaliable group by storage medium, and check is there are avaliable space for specify storage medium when create or alter table/partition, But the solution has a problem that it may cannot constrain the existing partition, so, we should separate the FE's High Watermark into two aspect, for example hdd_storage_high_watermark_usage_percent && ssd_storage_high_watermark_usage_percent?
2. should we refacor the checkDataSizeQuota logic to limit the quota by real usage?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo commented on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
WingsGo commented on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-619870711
I mean reject the operation and give the user error msg and solutions
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo commented on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
WingsGo commented on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-620555771
> > I mean reject the operation and give the user error msg and solutions
>
> I agreed. I've met this situation that user create table with SSD, but there is no SSD in cluster, this will cause lots of unnecessary migration tasks.
Yes, we meet the same situation, I will do this work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo commented on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
WingsGo commented on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-621145565
@morningman I found that CheckDataSizeQuota will only check by logic but not check by real disk usage.I have some questions:
1. One solution is get real disk space avaliable group by storage medium, and check is there are avaliable space for specify storage medium when create or alter table/partition, But the solution has a problem that it may cannot constrain the existing partition, and I don't know if the already exist partition exceed the limit cause by data loading, how doris deal with it or just ignore the situation, because stream load also doesn't limited by the db quota in fact.
2. should we refacor the checkDataSizeQuota logic to limit the quota by real usage?And if the real disk space is not insufficient, how doris deal with it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo edited a comment on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
WingsGo edited a comment on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-621145565
Hi, @morningman I found that CheckDataSizeQuota will only check by logic but not check by real disk usage. And I have some questions:
1. One solution is get real disk space avaliable group by storage medium, and check is there are avaliable space for specify storage medium when create or alter table/partition, But the solution has a problem that it may cannot constrain the existing partition, so should we separate the doris disk manage into two aspect, such as separate FE's High Watermark into two aspect like hdd_storage_high_watermark_usage_percent && ssd_storage_high_watermark_usage_percent?
2. Should we refacor the checkDataSizeQuota logic to limit the quota by real usage?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #3402: Can we give user a error msg when they create a olap table with medium which is not exists in cluster?
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #3402:
URL: https://github.com/apache/incubator-doris/issues/3402#issuecomment-620018535
> I mean reject the operation and give the user error msg and solutions
I agreed. I've met this situation that user create table with SSD, but there is no SSD in cluster, this will cause lots of unnecessary migration tasks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org