You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/11/02 09:33:00 UTC

[jira] [Commented] (KYLIN-3653) After kylin configured with hive data source with beeline, build failed if two jobs for creating flat table submitted at same time

    [ https://issues.apache.org/jira/browse/KYLIN-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672813#comment-16672813 ] 

ASF subversion and git services commented on KYLIN-3653:
--------------------------------------------------------

Commit caf109ea016c0bcbbef8f15c26b620003f253955 in kylin's branch refs/heads/master from Ni Chunen
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=caf109e ]

KYLIN-3653 Build failed if two jobs for creating flat table submitted at same time


> After kylin configured with hive data source with beeline, build failed if two jobs for creating flat table submitted at same time
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-3653
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3653
>             Project: Kylin
>          Issue Type: Bug
>          Components: Job Engine
>    Affects Versions: v2.4.1
>            Reporter: nichunen
>            Assignee: nichunen
>            Priority: Major
>             Fix For: v2.6.0
>
>
> The root cause is in org.apache.kylin.common.util.HiveCmdBuilder#build
> {code:java}
>         case BEELINE:
>             String tmpHqlPath = null;
>             StringBuilder hql = new StringBuilder();
>             try {
>                 tmpHqlPath = "/tmp/" + System.currentTimeMillis() + ".hql";
>                 for (String statement : statements) {
>                     hql.append(statement);
>                     hql.append("\n");
>                 }
>                 String createFileCmd = String.format(CREATE_HQL_TMP_FILE_TEMPLATE, tmpHqlPath, hql);
>                 buf.append(createFileCmd);
>                 buf.append("\n");
>                 buf.append(beelineShell);
>                 buf.append(" ");
>                 buf.append(beelineParams);
>                 buf.append(parseProps());
>                 buf.append(" -f ");
>                 buf.append(tmpHqlPath);
>                 buf.append(";ret_code=$?;rm -f ");
>                 buf.append(tmpHqlPath);
>                 buf.append(";exit $ret_code");
>             } finally {
>                 if (tmpHqlPath != null && logger.isDebugEnabled()) {
>                     logger.debug("The SQL to execute in beeline: \n" + hql);
>                 }
>             }
>             break;
> {code}
> A temp hql file will be created when creating flat table, the file name is the current time stamp. But if two jobs triggered with same time, there will be only one file, and jobs may be failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)