Kylin 2.5.3: metadata location vs. HIVE external tables

[Frédéric SOUCHU <Fr...@ingenico.com>]

Hi,
I've filled a similar question under: https://issues.apache.org/jira/browse/KYLIN-3989

I suspect my issue is not MapR related but rather a configuration issue - I don't see how Kylin can perform both:

  *   Store metadata as a HBase table named: maprfs:///apps/kylin_metadata
  *   Create temporary HIVE table using the same metadata path, e.g:


CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e

(
         PAYMENTS_GLOBALMERCHANTUID int,
         ...,
         ...
)
STORED AS SEQUENCEFILE
LOCATION 'maprfs:///apps/kylin_metadata/kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';

Why is the same path being used for both? HIVE cannot create a table within a HBase table using that syntax (see: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-Usage)

Is there a configuration property for HIVE LOCATION statements?

Regards,
Frederic
This email and its content belong to Ingenico Group. The enclosed information is confidential and may not be disclosed to any unauthorized person. If you have received it by mistake do not forward it and delete it from your system. Cet email et son contenu sont la propriété du Groupe Ingenico. L'information qu'il contient est confidentielle et ne peut être communiquée à des personnes non autorisées. Si vous l'avez reçu par erreur ne le transférez pas et supprimez-le.

Re: Kylin 2.5.3: metadata location vs. HIVE external tables

[Iñigo Martínez <im...@telecoming.com>]

Hi Frederic.

Kylin creates hive temporary tables at this property path:
 kylin.env.hdfs-working-dir

You can use an absolute or a relative path, that is:
kylin.env.hdfs-working-dir=/kylin/
or
kylin.env.hdfs-working-dir=hdfs://namenode/kylin/
or (not tested, we use HDP although it worked with s3:// in our EMR cluster
)
kylin.env.hdfs-working-dir=maprfs:///kylin/

If hive is using maprfs:///apps/kylin_metadata is because this property is
not set or is misconfigured.


El mar., 7 may. 2019 a las 9:07, Frédéric SOUCHU (<
Frederic.SOUCHU@ingenico.com>) escribió:

> Hi,
>
> I’ve filled a similar question under:
> https://issues.apache.org/jira/browse/KYLIN-3989
>
>
>
> I suspect my issue is not MapR related but rather a configuration issue –
> I don’t see how Kylin can perform both:
>
>    - Store metadata as a HBase table named: maprfs:///apps/kylin_metadata
>    - Create temporary HIVE table using *the same metadata path*, e.g:
>
>
>
> CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e
>
> (
>
>          PAYMENTS_GLOBALMERCHANTUID int,
>
>          ...,
>
>          ...
>
> )
>
> STORED AS SEQUENCEFILE
>
> LOCATION 'maprfs:///apps/kylin_metadata
> /kylin-15b76d79-29ce-5782-d84e-bd33c305fc6f/kylin_intermediate_txn_cube_99023bdd_79e8_1186_2480_28b1d352d09e';
>
>
>
> Why is the same path being used for both? HIVE cannot create a table
> within a HBase table using that syntax (see:
> https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-Usage
> )
>
>
>
> Is there a configuration property for HIVE LOCATION statements?
>
>
>
> Regards,
>
> Frederic
> This email and its content belong to Ingenico Group. The enclosed
> information is confidential and may not be disclosed to any unauthorized
> person. If you have received it by mistake do not forward it and delete it
> from your system. Cet email et son contenu sont la propriété du Groupe
> Ingenico. L’information qu’il contient est confidentielle et ne peut être
> communiquée à des personnes non autorisées. Si vous l’avez reçu par erreur
> ne le transférez pas et supprimez-le.
>


-- 




Iñigo Martínez
Systems Manager
imartinez@telecoming.com






  [image: Telecoming - Make it digital]
[image: 5000_empresas]
[image: 1000_empresas]

Paseo de la Castellana, 95. Torre Europa, pl 16. 28046 Madrid, Spain |
telecoming.com <http://www.telecoming.com/>



  Este correo electrónico y sus archivos adjuntos están dirigidos
únicamente a la(s) dirección(es) indicada(s) anteriormente. El carácter
confidencial, personal e intransferible del mismo está protegido
legalmente. Cualquier publicación, reproducción, distribución o
retransmisión no autorizada, ya sea completa o en parte, se encuentra
prohibida. Si ha recibido este mensaje por equivocación, notifíquelo
inmediatamente a la persona que lo ha enviado y borre el mensaje original
junto con sus ficheros anexos sin leerlo ni grabarlo en modo alguno.