You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by S*ST非烟飞雾 <85...@qq.com> on 2019/03/04 08:25:34 UTC
Dose joint dim constrained by Max Dimension Combination
hi all:
If i have a agg with joint dim has 4-dim, and i set the agg Max Dimension Combination=3. So will the result cuboid contain the 4-dim joint?
I find when i set the Max Dimension Combination with 3. the result cuboid contain some dim>3.
|---- Cuboid 111111111111111111111111111111111111111111111111111, est row: 34884166, est MB: 848.34 |---- Cuboid 101111111110010000111111111111111110001000000000011, est row: 31152743, est MB: 512.49, shrink: 89.3% |---- Cuboid 100000111110010000111111111111111110001000000000011, est row: 31305812, est MB: 485.15, shrink: 100.49% |---- Cuboid 100000000010010000111111111111111110001000000000011, est row: 26002267, est MB: 371.97, shrink: 83.06% |---- Cuboid 100000000000000000001111111111111110001000000000011, est row: 18314746, est MB: 240.16, shrink: 70.44% |---- Cuboid 100000000000000000000000111111111110001000000000011, est row: 17152263, est MB: 204.47, shrink: 93.65% |---- Cuboid 100000000000000000000000000101111110001000000000011, est row: 11100040, est MB: 116.44, shrink: 64.71% |---- Cuboid 100000000000000000000000000000001110001000000000011, est row: 143320, est MB: 1.26, shrink: 1.29% |---- Cuboid 100000000000000000000000000000000000000000000000011, est row: 281, est MB: 0, shrink: 0.2% |---- Cuboid 100000000000000000000000000000000000000000000000001, est row: 22, est MB: 0, shrink: 7.83% |---- Cuboid 100000000000000000000000000000000000000000000000010, est row: 30, est MB: 0, shrink: 10.68%
回复：Dose joint dim constrained by Max Dimension Combination
Posted by S*ST非烟飞雾 <85...@qq.com>.
i'v find it.
it's in
```
// 3. fill padding cuboids Map<Long, List<Long>> parent2Child = Maps.newHashMap(); Queue<Long> cuboidScan = new ArrayDeque<>(); cuboidScan.addAll(cuboidHolder); while (!cuboidScan.isEmpty()) { long current = cuboidScan.poll(); long parent = getParentOnPromise(current, cuboidHolder, forward); if (parent > 0) { if (!cuboidHolder.contains(parent)) { cuboidScan.add(parent); cuboidHolder.add(parent); } if (parent2Child.containsKey(parent)) { parent2Child.get(parent).add(current); } else { parent2Child.put(parent, Lists.newArrayList(current)); } } }
```
------------------ 原始邮件 ------------------
发件人: "S*ST非烟飞雾"<85...@qq.com>;
发送时间: 2019年3月6日(星期三) 晚上7:56
收件人: "user"<us...@kylin.apache.org>;
主题: 回复：Dose joint dim constrained by Max Dimension Combination
hi,
thanks for your reply. i have read the src previousely, and find it is build from child to parent until basecuboid. function `getOnTreeParentsByLayer` has filter all the cuboid that larger than maxdim, so i wonder which function add the padding cuboids as you mantioned in point 3?
**3, fill padding cuboids, which contain the parent of the cuboid in 2.(dim-count can more than Max Dimension Combination)**
```
private Set<Long> getOnTreeParentsByLayer(Collection<Long> children) {
Set<Long> parents = new HashSet<>();
for (long child : children) {
parents.addAll(getOnTreeParents(child));
}
parents = Sets.newHashSet(Iterators.filter(parents.iterator(), new Predicate<Long>() {
@Override
public boolean apply(@Nullable Long cuboidId) {
if (cuboidId == Cuboid.getBaseCuboidId(cubeDesc)) {
return true;
}
for (AggregationGroup agg : cubeDesc.getAggregationGroups()) {
if (agg.isOnTree(cuboidId) && checkDimCap(agg, cuboidId)) {
return true;
}
}
return false;
}
}));
return parents;
}
```
------------------ 原始邮件 ------------------
发件人: "Chao Long"<wa...@qq.com>;
发送时间: 2019年3月6日(星期三) 下午5:22
收件人: "user"<us...@kylin.apache.org>;
主题: 回复：Dose joint dim constrained by Max Dimension Combination
Hi,
I check the code and find joint dim is constrained by Max Dimension Combination, so if you set Max Dimension Combination=3, the cuboid result will not contain the 4-dim joint. The cuboid contain more then 3-dim will be filtered.
The final cuobid result contain:
1, BaseCuboid.
2, cuboid whose dim-count is less than Max Dimension Combination.
3, fill padding cuboids, which contain the parent of the cuboid in 2.(dim-count can more than Max Dimension Combination)
------------------
Best Regards,
Chao Long
------------------ 原始邮件 ------------------
发件人: "S*ST非烟飞雾"<85...@qq.com>;
发送时间: 2019年3月4日(星期一) 下午4:25
收件人: "user"<us...@kylin.apache.org>;
主题: Dose joint dim constrained by Max Dimension Combination
hi all:
If i have a agg with joint dim has 4-dim, and i set the agg Max Dimension Combination=3. So will the result cuboid contain the 4-dim joint?
I find when i set the Max Dimension Combination with 3. the result cuboid contain some dim>3.
|---- Cuboid 111111111111111111111111111111111111111111111111111, est row: 34884166, est MB: 848.34 |---- Cuboid 101111111110010000111111111111111110001000000000011, est row: 31152743, est MB: 512.49, shrink: 89.3% |---- Cuboid 100000111110010000111111111111111110001000000000011, est row: 31305812, est MB: 485.15, shrink: 100.49% |---- Cuboid 100000000010010000111111111111111110001000000000011, est row: 26002267, est MB: 371.97, shrink: 83.06% |---- Cuboid 100000000000000000001111111111111110001000000000011, est row: 18314746, est MB: 240.16, shrink: 70.44% |---- Cuboid 100000000000000000000000111111111110001000000000011, est row: 17152263, est MB: 204.47, shrink: 93.65% |---- Cuboid 100000000000000000000000000101111110001000000000011, est row: 11100040, est MB: 116.44, shrink: 64.71% |---- Cuboid 100000000000000000000000000000001110001000000000011, est row: 143320, est MB: 1.26, shrink: 1.29% |---- Cuboid 100000000000000000000000000000000000000000000000011, est row: 281, est MB: 0, shrink: 0.2% |---- Cuboid 100000000000000000000000000000000000000000000000001, est row: 22, est MB: 0, shrink: 7.83% |---- Cuboid 100000000000000000000000000000000000000000000000010, est row: 30, est MB: 0, shrink: 10.68%
回复：Dose joint dim constrained by Max Dimension Combination
Posted by S*ST非烟飞雾 <85...@qq.com>.
hi,
thanks for your reply. i have read the src previousely, and find it is build from child to parent until basecuboid. function `getOnTreeParentsByLayer` has filter all the cuboid that larger than maxdim, so i wonder which function add the padding cuboids as you mantioned in point 3?
**3, fill padding cuboids, which contain the parent of the cuboid in 2.(dim-count can more than Max Dimension Combination)**
```
private Set<Long> getOnTreeParentsByLayer(Collection<Long> children) {
Set<Long> parents = new HashSet<>();
for (long child : children) {
parents.addAll(getOnTreeParents(child));
}
parents = Sets.newHashSet(Iterators.filter(parents.iterator(), new Predicate<Long>() {
@Override
public boolean apply(@Nullable Long cuboidId) {
if (cuboidId == Cuboid.getBaseCuboidId(cubeDesc)) {
return true;
}
for (AggregationGroup agg : cubeDesc.getAggregationGroups()) {
if (agg.isOnTree(cuboidId) && checkDimCap(agg, cuboidId)) {
return true;
}
}
return false;
}
}));
return parents;
}
```
------------------ 原始邮件 ------------------
发件人: "Chao Long"<wa...@qq.com>;
发送时间: 2019年3月6日(星期三) 下午5:22
收件人: "user"<us...@kylin.apache.org>;
主题: 回复：Dose joint dim constrained by Max Dimension Combination
Hi,
I check the code and find joint dim is constrained by Max Dimension Combination, so if you set Max Dimension Combination=3, the cuboid result will not contain the 4-dim joint. The cuboid contain more then 3-dim will be filtered.
The final cuobid result contain:
1, BaseCuboid.
2, cuboid whose dim-count is less than Max Dimension Combination.
3, fill padding cuboids, which contain the parent of the cuboid in 2.(dim-count can more than Max Dimension Combination)
------------------
Best Regards,
Chao Long
------------------ 原始邮件 ------------------
发件人: "S*ST非烟飞雾"<85...@qq.com>;
发送时间: 2019年3月4日(星期一) 下午4:25
收件人: "user"<us...@kylin.apache.org>;
主题: Dose joint dim constrained by Max Dimension Combination
hi all:
If i have a agg with joint dim has 4-dim, and i set the agg Max Dimension Combination=3. So will the result cuboid contain the 4-dim joint?
I find when i set the Max Dimension Combination with 3. the result cuboid contain some dim>3.
|---- Cuboid 111111111111111111111111111111111111111111111111111, est row: 34884166, est MB: 848.34 |---- Cuboid 101111111110010000111111111111111110001000000000011, est row: 31152743, est MB: 512.49, shrink: 89.3% |---- Cuboid 100000111110010000111111111111111110001000000000011, est row: 31305812, est MB: 485.15, shrink: 100.49% |---- Cuboid 100000000010010000111111111111111110001000000000011, est row: 26002267, est MB: 371.97, shrink: 83.06% |---- Cuboid 100000000000000000001111111111111110001000000000011, est row: 18314746, est MB: 240.16, shrink: 70.44% |---- Cuboid 100000000000000000000000111111111110001000000000011, est row: 17152263, est MB: 204.47, shrink: 93.65% |---- Cuboid 100000000000000000000000000101111110001000000000011, est row: 11100040, est MB: 116.44, shrink: 64.71% |---- Cuboid 100000000000000000000000000000001110001000000000011, est row: 143320, est MB: 1.26, shrink: 1.29% |---- Cuboid 100000000000000000000000000000000000000000000000011, est row: 281, est MB: 0, shrink: 0.2% |---- Cuboid 100000000000000000000000000000000000000000000000001, est row: 22, est MB: 0, shrink: 7.83% |---- Cuboid 100000000000000000000000000000000000000000000000010, est row: 30, est MB: 0, shrink: 10.68%
回复：Dose joint dim constrained by Max Dimension Combination
Posted by Chao Long <wa...@qq.com>.
Hi,
I check the code and find joint dim is constrained by Max Dimension Combination, so if you set Max Dimension Combination=3, the cuboid result will not contain the 4-dim joint. The cuboid contain more then 3-dim will be filtered.
The final cuobid result contain:
1, BaseCuboid.
2, cuboid whose dim-count is less than Max Dimension Combination.
3, fill padding cuboids, which contain the parent of the cuboid in 2.(dim-count can more than Max Dimension Combination)
------------------
Best Regards,
Chao Long
------------------ 原始邮件 ------------------
发件人: "S*ST非烟飞雾"<85...@qq.com>;
发送时间: 2019年3月4日(星期一) 下午4:25
收件人: "user"<us...@kylin.apache.org>;
主题: Dose joint dim constrained by Max Dimension Combination
hi all:
If i have a agg with joint dim has 4-dim, and i set the agg Max Dimension Combination=3. So will the result cuboid contain the 4-dim joint?
I find when i set the Max Dimension Combination with 3. the result cuboid contain some dim>3.
|---- Cuboid 111111111111111111111111111111111111111111111111111, est row: 34884166, est MB: 848.34 |---- Cuboid 101111111110010000111111111111111110001000000000011, est row: 31152743, est MB: 512.49, shrink: 89.3% |---- Cuboid 100000111110010000111111111111111110001000000000011, est row: 31305812, est MB: 485.15, shrink: 100.49% |---- Cuboid 100000000010010000111111111111111110001000000000011, est row: 26002267, est MB: 371.97, shrink: 83.06% |---- Cuboid 100000000000000000001111111111111110001000000000011, est row: 18314746, est MB: 240.16, shrink: 70.44% |---- Cuboid 100000000000000000000000111111111110001000000000011, est row: 17152263, est MB: 204.47, shrink: 93.65% |---- Cuboid 100000000000000000000000000101111110001000000000011, est row: 11100040, est MB: 116.44, shrink: 64.71% |---- Cuboid 100000000000000000000000000000001110001000000000011, est row: 143320, est MB: 1.26, shrink: 1.29% |---- Cuboid 100000000000000000000000000000000000000000000000011, est row: 281, est MB: 0, shrink: 0.2% |---- Cuboid 100000000000000000000000000000000000000000000000001, est row: 22, est MB: 0, shrink: 7.83% |---- Cuboid 100000000000000000000000000000000000000000000000010, est row: 30, est MB: 0, shrink: 10.68%