You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "JackDrogon (via GitHub)" <gi...@apache.org> on 2023/03/09 11:14:52 UTC
[GitHub] [doris] JackDrogon opened a new pull request, #17622: [Improvement](dynamic partition) Support bigint unix timestamp as par…
JackDrogon opened a new pull request, #17622:
URL: https://github.com/apache/doris/pull/17622
## Problem summary
可以把unix timestamp的值当做动态分区的分区键,时间计算是把它当做unix timestamp来算
PR添加了一个property `dynamic_partition.time_source`, 默认是空,是原先的行为。如果为`unix_timestamp`就把它当做unix时间戳使用. 分区range不再是date转化为字符串,还是unix timestamp
建表语句
```sql
CREATE TABLE tbl_day
(
k1 bigint
)
PARTITION BY RANGE(k1) ()
DISTRIBUTED BY HASH(k1)
PROPERTIES
(
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.time_source" = "unix_timestamp",
"dynamic_partition.start" = "-2",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "8",
"replication_allocation" = "tag.location.default: 1"
);
```
```bash
mysql> show partitions from tbl_day;
+-------------+---------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation | IsMutable |
+-------------+---------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
| 61055 | p20230308 | 1 | 2023-03-08 17:20:06 | NORMAL | k1 | [types: [BIGINT]; keys: [1678204800]; ..types: [BIGINT]; keys: [1678291200]; ) | k1 | 8 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 | true |
| 61072 | p20230309 | 1 | 2023-03-08 17:20:06 | NORMAL | k1 | [types: [BIGINT]; keys: [1678291200]; ..types: [BIGINT]; keys: [1678377600]; ) | k1 | 8 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 | true |
| 61089 | p20230310 | 1 | 2023-03-08 17:20:06 | NORMAL | k1 | [types: [BIGINT]; keys: [1678377600]; ..types: [BIGINT]; keys: [1678464000]; ) | k1 | 8 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 | true |
| 61106 | p20230311 | 1 | 2023-03-09 00:09:12 | NORMAL | k1 | [types: [BIGINT]; keys: [1678464000]; ..types: [BIGINT]; keys: [1678550400]; ) | k1 | 8 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 | true |
+-------------+---------------+----------------+---------------------+--------+--------------+--------------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+-----------+
```
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] JackDrogon commented on pull request #17622: [Improvement](dynamic partition) Support bigint unix timestamp as par…
Posted by "JackDrogon (via GitHub)" <gi...@apache.org>.
JackDrogon commented on PR #17622:
URL: https://github.com/apache/doris/pull/17622#issuecomment-1463169105
> ` "dynamic_partition.time_source" = "unix_timestamp"` is unnecessary
before support this, doris support bigint as partition key, but it use range like [20230306, 20230313), not convert it to timestamp just like a date string. To be compatible with the behavior of older versions,I add time_source
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](dynamic partition) Support bigint unix timestamp as par… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #17622: [Improvement](dynamic partition) Support bigint unix timestamp as par…
URL: https://github.com/apache/doris/pull/17622
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Improvement](dynamic partition) Support bigint unix timestamp as par… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17622:
URL: https://github.com/apache/doris/pull/17622#issuecomment-1791725883
We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #17622: [Improvement](dynamic partition) Support bigint unix timestamp as par…
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #17622:
URL: https://github.com/apache/doris/pull/17622#issuecomment-1473196589
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on pull request #17622: [Improvement](dynamic partition) Support bigint unix timestamp as par…
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #17622:
URL: https://github.com/apache/doris/pull/17622#issuecomment-1463060375
` "dynamic_partition.time_source" = "unix_timestamp"` is unnecessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #17622: [Improvement](dynamic partition) Support bigint unix timestamp as par…
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on code in PR #17622:
URL: https://github.com/apache/doris/pull/17622#discussion_r1133418498
##########
fe/fe-core/src/main/java/org/apache/doris/common/ErrorCode.java:
##########
@@ -22,1006 +22,1016 @@
// Error code used to indicate what error happened.
public enum ErrorCode {
// Try our best to compatible with MySQL's
- ERR_HASHCHK(1000, new byte[]{'H', 'Y', '0', '0', '0'}, "hashchk"),
- ERR_DB_CREATE_EXISTS(1007, new byte[]{'H', 'Y', '0', '0', '0'}, "Can't create database '%s'; database exists"),
- ERR_DB_DROP_EXISTS(1008, new byte[]{'H', 'Y', '0', '0', '0'}, "Can't drop database '%s'; database doesn't exist"),
- ERR_DBACCESS_DENIED_ERROR(1044, new byte[]{'4', '2', '0', '0', '0'}, "Access denied for user '%s' to "
+ ERR_HASHCHK(1000, new byte[] {'H', 'Y', '0', '0', '0'}, "hashchk"),
Review Comment:
Better not format the code not changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org