You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "YongHun Jeon (JIRA)" <ji...@apache.org> on 2014/06/19 04:43:24 UTC
[jira] [Created] (HADOOP-10721) The result does not show up after
running hive query on Swift.
YongHun Jeon created HADOOP-10721:
-------------------------------------
Summary: The result does not show up after running hive query on Swift.
Key: HADOOP-10721
URL: https://issues.apache.org/jira/browse/HADOOP-10721
Project: Hadoop Common
Issue Type: Bug
Components: fs/swift
Reporter: YongHun Jeon
Priority: Critical
I configured Hadoop and Swift system as the site is mentioned : http://docs.openstack.org/developer/sahara/userdoc/hadoop-swift.html.
So, I succeeded to access the Swift from Hadoop.
I am running TPC-H performance test on Hadoop system integrated with Swift.
I ran the below hive query.
---------------------------------------------------------------------------------------------
DROP TABLE lineitem;
DROP TABLE q1_pricing_summary_report;
-- create tables and load data
Create external table lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, L_SHIPDATE STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS TEXTFILE LOCATION 'swift://test.provider/tpch/lineitem';
-- create the target table
CREATE external TABLE q1_pricing_summary_report ( L_RETURNFLAG STRING, L_LINESTATUS STRING, SUM_QTY DOUBLE, SUM_BASE_PRICE DOUBLE, SUM_DISC_PRICE DOUBLE, SUM_CHARGE DOUBLE, AVE_QTY DOUBLE, AVE_PRICE DOUBLE, AVE_DISC DOUBLE, COUNT_ORDER INT) LOCATION 'swift://test.provider/user/result/q1_pricing_summary_report';
set mapred.min.split.size=536870912;
-- the query
INSERT OVERWRITE TABLE q1_pricing_summary_report
SELECT
L_RETURNFLAG, L_LINESTATUS, SUM(L_QUANTITY), SUM(L_EXTENDEDPRICE), SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)), SUM(L_EXTENDEDPRICE*(1-L_DISCOUNT)*(1+L_TAX)), AVG(L_QUANTITY), AVG(L_EXTENDEDPRICE), AVG(L_DISCOUNT), COUNT(1)
FROM
lineitem
WHERE
L_SHIPDATE<='1998-09-02'
GROUP BY L_RETURNFLAG, L_LINESTATUS
ORDER BY L_RETURNFLAG, L_LINESTATUS;
---------------------------------------------------------------------------------------------
You can get the files(such as lineitem) for the test through running dbgen which is in this site : http://www.tpc.org/tpch/.
I saw the some temporary files are generated and deleted. However, the result does not show up after running hive query.
--
This message was sent by Atlassian JIRA
(v6.2#6252)