You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Roberto Tardío Olmos <ro...@stratebi.com> on 2017/04/23 19:52:18 UTC
Issues with Kylin 1.6 and Tableau 10
Hi Kylin community,
I'm trying to analyze data from Kylin 1.6 cube using Tableau 10.2
desktop. Following the guide published for Tableau 9 at
http://kylin.apache.org/docs20/tutorial/tableau_91.html , I can connect
and perform some analysis over data. However I have experimented the
following annoying issues:
1. I drag and drop the fact table (about 100 million rows) to the data
sources canvas. Then, when I click on "Update Now" button sometimes
Tableau and Kylin crash, after a while waiting for data source query
dialog.
* When it happens Kylin log just show the following warning:
o 2017-04-21 15:37:30,353 WARN[Query
c01e6b99-557f-4453-ada1-46bd8845fa10-154]
sizeof.ObjectGraphWalker:209 : The configured limit of 1.000
object references was reached while attempting to calculate
the size of the object graph. Severe performance degradation
could occur if the sizing operation continues. This can be
avoided by setting the CacheManger or Cache <sizeOfPolicy>
elements maxDepthExceededBehavior to "abort" or adding stop
points with @IgnoreSizeOf annotations. If performance
degradation is NOT an issue at the configured limit, raise
the limit value using the CacheManager or Cache
<sizeOfPolicy> elements maxDepth attribute. For more
information, see the Ehcache configuration documentation.
* The above issue never happens with dimension tables. Maybe due
to they have lower cardinality than fact fable.
2. The above issue does not always happen and I was able to develop a
dashboard with my Kylin Data Source. However I also experimented the
same issue when I open my saved Tableau project with Kylin
dashboard. In this case, Tableau can not connect to data source and
Kylin crash.
3. The above issue never happens when I define a custom SQL for fact
table. However this only goes well when I use Custom SQL without
joins. That way lead to bad queries If i try to perform joins
between fact (custom sql) and dimension tables.
* For example, the following bad query is generated by Tableau and
seems not supported by Kylin.
o SELECT "F_RENDIMIENTO_VT"."CREDITOS" AS "CREDITOS",
"F_RENDIMIENTO_VT"."FLG_CALCULO_TASAS" AS
"FLG_CALCULO_TASAS",
"F_RENDIMIENTO_VT"."FLG_MATRICULADA" AS "FLG_MATRICULADA",
"F_RENDIMIENTO_VT"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
"F_RENDIMIENTO_VT"."FLG_SUPERADA" AS "FLG_SUPERADA",
"F_RENDIMIENTO_VT"."ID_CURSO_ACADEMICO" AS
"ID_CURSO_ACADEMICO",
"F_RENDIMIENTO_VT"."ID_PLAN_ESTUDIO" AS "ID_PLAN_ESTUDIO",
"F_RENDIMIENTO_VT"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
"D_RANGO_EDAD_VK"."ID_RANGO_EDAD" AS
"ID_RANGO_EDAD__D_RANGO_EDAD_VK_",
"F_RENDIMIENTO_VT"."ID_SEXO" AS "ID_SEXO",
"D_RANGO_EDAD_VK"."NOMBRE_RANGO_EDAD" AS "NOMBRE_RANGO_EDAD",
1 AS "Number_of_Records"
FROM (
SELECT "F_RENDIMIENTO"."CREDITOS" AS "CREDITOS",
"F_RENDIMIENTO"."FLG_CALCULO_TASAS" AS "FLG_CALCULO_TASAS",
"F_RENDIMIENTO"."FLG_MATRICULADA" AS "FLG_MATRICULADA",
"F_RENDIMIENTO"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
"F_RENDIMIENTO"."FLG_SUPERADA" AS "FLG_SUPERADA",
"F_RENDIMIENTO"."ID_CURSO_ACADEMICO" AS
"ID_CURSO_ACADEMICO",
"F_RENDIMIENTO"."ID_PLAN_ESTUDIO" AS "ID_PLAN_ESTUDIO",
"F_RENDIMIENTO"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
"F_RENDIMIENTO"."ID_SEXO" AS "ID_SEXO"
FROM "DM_ACAD_KYLIN_ORC"."F_RENDIMIENTO" "F_RENDIMIENTO"
LIMIT 10000
) "F_RENDIMIENTO_VT"
INNER JOIN "DM_ACAD_KYLIN_ORC"."D_RANGO_EDAD_VK"
"D_RANGO_EDAD_VK" ON ("F_RENDIMIENTO_VT"."ID_RANGO_EDAD" =
"D_RANGO_EDAD_VK"."ID_RANGO_EDAD")
LIMIT 1000
*�Has someone experimented similar issues with Tableau 10.2 or previous
versions? **
*
*�Any solutions or other workarounds?*
Kind regards, I will appreciate your help
--
*Roberto Tard�o Olmos*
/Senior Big Data & Business Intelligence Consultant/
Avenida de Brasil, 17, Planta 16.
28020 Madrid
Fijo: 91.788.34.10
Re: Issues with Kylin 1.6 and Tableau 10
Posted by Roberto Tardío Olmos <ro...@stratebi.com>.
Thanks Billy,
Effectively, I have checked "Live" but Tableau is launching the
following query as I can see in Kylin log:
SELECT * FROM "DM_ACAD_KYLIN_ORC"."F_RENDIMIENTO"
without the LIMIT clause. Surprisingly, when Tableau show error info
about query It shows that the query launched is the following, including
the limit clause:
SELECT "F_RENDIMIENTO"."CREDITOS" AS "CREDITOS",
"F_RENDIMIENTO"."FLG_CALCULO_TASAS" AS "FLG_CALCULO_TASAS",
"F_RENDIMIENTO"."FLG_MATRICULADA" AS "FLG_MATRICULADA",
"F_RENDIMIENTO"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
"F_RENDIMIENTO"."FLG_SUPERADA" AS "FLG_SUPERADA",
"F_RENDIMIENTO"."ID_CURSO_ACADEMICO" AS "ID_CURSO_ACADEMICO",
"F_RENDIMIENTO"."ID_PLAN_ESTUDIO" AS "ID_PLAN_ESTUDIO",
"F_RENDIMIENTO"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
"F_RENDIMIENTO"."ID_SEXO" AS "ID_SEXO"
FROM "DM_ACAD_KYLIN_ORC"."F_RENDIMIENTO" "F_RENDIMIENTO"
LIMIT 10000
I checked that this occurs when I drag any kind of table, but Dimension
table cardinality is very small to crash. Surprisingly Tableau first
launches SELECT * query and after launches another query with the
appropriate limit clause.
1. First a query Select * From Table is launched.
* 2017-04-25 12:10:02,785 INFO [Query
10a8ed73-5517-45b7-a178-42e241685eb9-52]
service.QueryService:336 : Using project: DEMO_2_KYLIN_STRATEBI
2017-04-25 12:10:02,785 INFO [Query
10a8ed73-5517-45b7-a178-42e241685eb9-52] service.QueryService:337 :
The original query: SELECT * FROM
"DM_ACAD_KYLIN_ORC"."D_CURSO_ACADEMICO_VK"
2. After that the correct query with LIMIT clause is launched.
* SELECT "D_ESTUDIO_VK"."ID_PLAN_ESTUDIO" AS "ID_PLAN_ESTUDIO",
"D_ESTUDIO_VK"."ID_TIPO_ESTUDIO" AS "ID_TIPO_ESTUDIO",
"D_ESTUDIO_VK"."NOMBRE_PLAN_ESTUDIO" AS "NOMBRE_PLAN_ESTUDIO",
"D_ESTUDIO_VK"."NOMBRE_TIPO_ESTUDIO" AS "NOMBRE_TIPO_ESTUDIO",
1 AS "Number_of_Records" FROM "DM_ACAD_KYLIN_ORC"."D_ESTUDIO_VK"
"D_ESTUDIO_VK"*LIMIT 1000*
�What do you think? Looks like a bug of Tableau 10.2.
Regards,
El 24/04/2017 a las 15:36, Billy Liu escribi�:
> Kylin crashes since Tableau issues the "select * from fact_table",
> that query will load all fact records and exhaust all memory resource.
> Have you checked the "Live" connection on the Tableau? In \u201cLive" mode,
> Tableau should not send out that query.
>
> 2017-04-24 3:52 GMT+08:00 Roberto Tard�o Olmos
> <roberto.tardio@stratebi.com <ma...@stratebi.com>>:
>
> Hi Kylin community,
>
> I'm trying to analyze data from Kylin 1.6 cube using Tableau 10.2
> desktop. Following the guide published for Tableau 9 at
> http://kylin.apache.org/docs20/tutorial/tableau_91.html
> <http://kylin.apache.org/docs20/tutorial/tableau_91.html> , I can
> connect and perform some analysis over data. However I have
> experimented the following annoying issues:
>
> 1. I drag and drop the fact table (about 100 million rows) to the
> data sources canvas. Then, when I click on "Update Now" button
> sometimes Tableau and Kylin crash, after a while waiting for
> data source query dialog.
> * When it happens Kylin log just show the following warning:
> o 2017-04-21 15:37:30,353 WARN[Query
> c01e6b99-557f-4453-ada1-46bd8845fa10-154]
> sizeof.ObjectGraphWalker:209 : The configured limit of
> 1.000 object references was reached while attempting
> to calculate the size of the object graph. Severe
> performance degradation could occur if the sizing
> operation continues. This can be avoided by setting
> the CacheManger or Cache <sizeOfPolicy> elements
> maxDepthExceededBehavior to "abort" or adding stop
> points with @IgnoreSizeOf annotations. If performance
> degradation is NOT an issue at the configured limit,
> raise the limit value using the CacheManager or Cache
> <sizeOfPolicy> elements maxDepth attribute. For more
> information, see the Ehcache configuration documentation.
> * The above issue never happens with dimension tables. Maybe
> due to they have lower cardinality than fact fable.
> 2. The above issue does not always happen and I was able to
> develop a dashboard with my Kylin Data Source. However I also
> experimented the same issue when I open my saved Tableau
> project with Kylin dashboard. In this case, Tableau can not
> connect to data source and Kylin crash.
> 3. The above issue never happens when I define a custom SQL for
> fact table. However this only goes well when I use Custom SQL
> without joins. That way lead to bad queries If i try to
> perform joins between fact (custom sql) and dimension tables.
> * For example, the following bad query is generated by
> Tableau and seems not supported by Kylin.
> o SELECT "F_RENDIMIENTO_VT"."CREDITOS" AS "CREDITOS",
> "F_RENDIMIENTO_VT"."FLG_CALCULO_TASAS" AS
> "FLG_CALCULO_TASAS",
> "F_RENDIMIENTO_VT"."FLG_MATRICULADA" AS
> "FLG_MATRICULADA",
> "F_RENDIMIENTO_VT"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
> "F_RENDIMIENTO_VT"."FLG_SUPERADA" AS "FLG_SUPERADA",
> "F_RENDIMIENTO_VT"."ID_CURSO_ACADEMICO" AS
> "ID_CURSO_ACADEMICO",
> "F_RENDIMIENTO_VT"."ID_PLAN_ESTUDIO" AS
> "ID_PLAN_ESTUDIO",
> "F_RENDIMIENTO_VT"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
> "D_RANGO_EDAD_VK"."ID_RANGO_EDAD" AS
> "ID_RANGO_EDAD__D_RANGO_EDAD_VK_",
> "F_RENDIMIENTO_VT"."ID_SEXO" AS "ID_SEXO",
> "D_RANGO_EDAD_VK"."NOMBRE_RANGO_EDAD" AS
> "NOMBRE_RANGO_EDAD",
> 1 AS "Number_of_Records"
> FROM (
> SELECT "F_RENDIMIENTO"."CREDITOS" AS "CREDITOS",
> "F_RENDIMIENTO"."FLG_CALCULO_TASAS" AS
> "FLG_CALCULO_TASAS",
> "F_RENDIMIENTO"."FLG_MATRICULADA" AS
> "FLG_MATRICULADA",
> "F_RENDIMIENTO"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
> "F_RENDIMIENTO"."FLG_SUPERADA" AS "FLG_SUPERADA",
> "F_RENDIMIENTO"."ID_CURSO_ACADEMICO" AS
> "ID_CURSO_ACADEMICO",
> "F_RENDIMIENTO"."ID_PLAN_ESTUDIO" AS
> "ID_PLAN_ESTUDIO",
> "F_RENDIMIENTO"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
> "F_RENDIMIENTO"."ID_SEXO" AS "ID_SEXO"
> FROM "DM_ACAD_KYLIN_ORC"."F_RENDIMIENTO" "F_RENDIMIENTO"
> LIMIT 10000
> ) "F_RENDIMIENTO_VT"
> INNER JOIN "DM_ACAD_KYLIN_ORC"."D_RANGO_EDAD_VK"
> "D_RANGO_EDAD_VK" ON
> ("F_RENDIMIENTO_VT"."ID_RANGO_EDAD" =
> "D_RANGO_EDAD_VK"."ID_RANGO_EDAD")
> LIMIT 1000
>
> *�Has someone experimented similar issues with Tableau 10.2 or
> previous versions? **
> *
>
> *�Any solutions or other workarounds?*
>
> Kind regards, I will appreciate your help
>
> --
> *Roberto Tard�o Olmos*
> /Senior Big Data & Business Intelligence Consultant/
>
> Avenida de Brasil, 17, Planta 16.
>
> 28020 Madrid
>
> Fijo: 91.788.34.10
>
>
--
*Roberto Tard�o Olmos*
/Senior Big Data & Business Intelligence Consultant/
Avenida de Brasil, 17, Planta 16.
28020 Madrid
Fijo: 91.788.34.10
Re: Issues with Kylin 1.6 and Tableau 10
Posted by Billy Liu <bi...@apache.org>.
Kylin crashes since Tableau issues the "select * from fact_table", that
query will load all fact records and exhaust all memory resource. Have you
checked the "Live" connection on the Tableau? In “Live" mode, Tableau
should not send out that query.
2017-04-24 3:52 GMT+08:00 Roberto Tardío Olmos <ro...@stratebi.com>
:
> Hi Kylin community,
>
> I'm trying to analyze data from Kylin 1.6 cube using Tableau 10.2 desktop.
> Following the guide published for Tableau 9 at http://kylin.apache.org/
> docs20/tutorial/tableau_91.html , I can connect and perform some analysis
> over data. However I have experimented the following annoying issues:
>
> 1. I drag and drop the fact table (about 100 million rows) to the data
> sources canvas. Then, when I click on "Update Now" button sometimes Tableau
> and Kylin crash, after a while waiting for data source query dialog.
> - When it happens Kylin log just show the following warning:
> - 2017-04-21 15:37:30,353 WARN [Query c01e6b99-557f-4453-ada1-46bd8845fa10-154]
> sizeof.ObjectGraphWalker:209 : The configured limit of 1.000 object
> references was reached while attempting to calculate the size of the object
> graph. Severe performance degradation could occur if the sizing operation
> continues. This can be avoided by setting the CacheManger or Cache
> <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop
> points with @IgnoreSizeOf annotations. If performance degradation is NOT an
> issue at the configured limit, raise the limit value using the CacheManager
> or Cache <sizeOfPolicy> elements maxDepth attribute. For more information,
> see the Ehcache configuration documentation.
> - The above issue never happens with dimension tables. Maybe due to
> they have lower cardinality than fact fable.
> 2. The above issue does not always happen and I was able to develop a
> dashboard with my Kylin Data Source. However I also experimented the same
> issue when I open my saved Tableau project with Kylin dashboard. In this
> case, Tableau can not connect to data source and Kylin crash.
> 3. The above issue never happens when I define a custom SQL for fact
> table. However this only goes well when I use Custom SQL without joins.
> That way lead to bad queries If i try to perform joins between fact (custom
> sql) and dimension tables.
> - For example, the following bad query is generated by Tableau and
> seems not supported by Kylin.
> - SELECT "F_RENDIMIENTO_VT"."CREDITOS" AS "CREDITOS",
> "F_RENDIMIENTO_VT"."FLG_CALCULO_TASAS" AS "FLG_CALCULO_TASAS",
> "F_RENDIMIENTO_VT"."FLG_MATRICULADA" AS "FLG_MATRICULADA",
> "F_RENDIMIENTO_VT"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
> "F_RENDIMIENTO_VT"."FLG_SUPERADA" AS "FLG_SUPERADA",
> "F_RENDIMIENTO_VT"."ID_CURSO_ACADEMICO" AS
> "ID_CURSO_ACADEMICO",
> "F_RENDIMIENTO_VT"."ID_PLAN_ESTUDIO" AS "ID_PLAN_ESTUDIO",
> "F_RENDIMIENTO_VT"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
> "D_RANGO_EDAD_VK"."ID_RANGO_EDAD" AS
> "ID_RANGO_EDAD__D_RANGO_EDAD_VK_",
> "F_RENDIMIENTO_VT"."ID_SEXO" AS "ID_SEXO",
> "D_RANGO_EDAD_VK"."NOMBRE_RANGO_EDAD" AS "NOMBRE_RANGO_EDAD",
> 1 AS "Number_of_Records"
> FROM (
> SELECT "F_RENDIMIENTO"."CREDITOS" AS "CREDITOS",
> "F_RENDIMIENTO"."FLG_CALCULO_TASAS" AS "FLG_CALCULO_TASAS",
> "F_RENDIMIENTO"."FLG_MATRICULADA" AS "FLG_MATRICULADA",
> "F_RENDIMIENTO"."FLG_PRESENTADA" AS "FLG_PRESENTADA",
> "F_RENDIMIENTO"."FLG_SUPERADA" AS "FLG_SUPERADA",
> "F_RENDIMIENTO"."ID_CURSO_ACADEMICO" AS "ID_CURSO_ACADEMICO",
> "F_RENDIMIENTO"."ID_PLAN_ESTUDIO" AS "ID_PLAN_ESTUDIO",
> "F_RENDIMIENTO"."ID_RANGO_EDAD" AS "ID_RANGO_EDAD",
> "F_RENDIMIENTO"."ID_SEXO" AS "ID_SEXO"
> FROM "DM_ACAD_KYLIN_ORC"."F_RENDIMIENTO" "F_RENDIMIENTO"
> LIMIT 10000
> ) "F_RENDIMIENTO_VT"
> INNER JOIN "DM_ACAD_KYLIN_ORC"."D_RANGO_EDAD_VK"
> "D_RANGO_EDAD_VK" ON ("F_RENDIMIENTO_VT"."ID_RANGO_EDAD" =
> "D_RANGO_EDAD_VK"."ID_RANGO_EDAD")
> LIMIT 1000
>
> *¿Has someone experimented similar issues with Tableau 10.2 or previous
> versions? *
>
> *¿Any solutions or other workarounds?*
>
> Kind regards, I will appreciate your help
> --
> *Roberto Tardío Olmos*
> *Senior Big Data & Business Intelligence Consultant*
>
> Avenida de Brasil, 17, Planta 16. 28020 Madrid Fijo: 91.788.34.10
>