You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 耳东 <77...@qq.com> on 2016/05/16 16:25:09 UTC
Partition date column
Hi all :
When we create the model, there is a property called partition date column, and the format the web interface shows is yyyyMMdd or yyyy-MM-dd. Can this partition date column be a timestamp like 1463406381 or 1463415185143.
When we build the cube, there are two parameters called startTime and endTime ,and the format the instruction shows is 1388563200000 for 2014-1-1. Which column will be used to filter the data? If it use the partition date column, is there any tranformation in the kylin?
Re: Partition date column
Posted by Li Yang <li...@apache.org>.
> In my opinion, kylin can change the yyyy-MM-dd and yyyyMMdd to millis.
and the cardinality is equal to the cardinality of 'yyyyMMdd' or
yyyy-MM-dd.
Makes sense. The data format is a separate thing as to cardinality. By
rounding millis to hours to days, the column can still be low in
cardinality. I'm re-opening the JIRA 1698.
On Thu, May 19, 2016 at 5:03 PM, 耳东 <77...@qq.com> wrote:
> we have opened a JIRA. And the reponse is as follows:
> millis format as partition column will not be support in short run. The
> reason is that millis format tends to come with large cardinality, which is
> not acceptable by kylin. As a workaround you may use hive view to map the
> millis format to 'yyyy-MM-dd' or 'yyyyMMdd' format.
> In my opinion, kylin can change the yyyy-MM-dd and yyyyMMdd to millis.
> and the cardinality is equal to the cardinality of 'yyyyMMdd' or
> yyyy-MM-dd.
>
>
>
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Li Yang";<li...@apache.org>;
> 发送时间: 2016年5月19日(星期四) 下午4:49
> 收件人: "dev"<de...@kylin.apache.org>;
>
> 主题: Re: Partition date column
>
>
>
> Please open a JIRA for the new feature. It should not be too difficult to
> support long millis as partition date column.
>
> On Tue, May 17, 2016 at 12:25 AM, 耳东 <77...@qq.com> wrote:
>
> > Hi all :
> > When we create the model, there is a property called partition date
> > column, and the format the web interface shows is yyyyMMdd or yyyy-MM-dd.
> > Can this partition date column be a timestamp like 1463406381 or
> > 1463415185143.
> >
> >
> > When we build the cube, there are two parameters called startTime
> and
> > endTime ,and the format the instruction shows is 1388563200000 for
> > 2014-1-1. Which column will be used to filter the data? If it use the
> > partition date column, is there any tranformation in the kylin?
>
回复： Partition date column
Posted by 耳东 <77...@qq.com>.
we have opened a JIRA. And the reponse is as follows:
millis format as partition column will not be support in short run. The reason is that millis format tends to come with large cardinality, which is not acceptable by kylin. As a workaround you may use hive view to map the millis format to 'yyyy-MM-dd' or 'yyyyMMdd' format.
In my opinion, kylin can change the yyyy-MM-dd and yyyyMMdd to millis. and the cardinality is equal to the cardinality of 'yyyyMMdd' or yyyy-MM-dd.
------------------ 原始邮件 ------------------
发件人: "Li Yang";<li...@apache.org>;
发送时间: 2016年5月19日(星期四) 下午4:49
收件人: "dev"<de...@kylin.apache.org>;
主题: Re: Partition date column
Please open a JIRA for the new feature. It should not be too difficult to
support long millis as partition date column.
On Tue, May 17, 2016 at 12:25 AM, 耳东 <77...@qq.com> wrote:
> Hi all :
> When we create the model, there is a property called partition date
> column, and the format the web interface shows is yyyyMMdd or yyyy-MM-dd.
> Can this partition date column be a timestamp like 1463406381 or
> 1463415185143.
>
>
> When we build the cube, there are two parameters called startTime and
> endTime ,and the format the instruction shows is 1388563200000 for
> 2014-1-1. Which column will be used to filter the data? If it use the
> partition date column, is there any tranformation in the kylin?
Re: Partition date column
Posted by Li Yang <li...@apache.org>.
Please open a JIRA for the new feature. It should not be too difficult to
support long millis as partition date column.
On Tue, May 17, 2016 at 12:25 AM, 耳东 <77...@qq.com> wrote:
> Hi all :
> When we create the model, there is a property called partition date
> column, and the format the web interface shows is yyyyMMdd or yyyy-MM-dd.
> Can this partition date column be a timestamp like 1463406381 or
> 1463415185143.
>
>
> When we build the cube, there are two parameters called startTime and
> endTime ,and the format the instruction shows is 1388563200000 for
> 2014-1-1. Which column will be used to filter the data? If it use the
> partition date column, is there any tranformation in the kylin?