You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@eagle.apache.org by RalphSu <gi...@git.apache.org> on 2015/12/14 09:52:38 UTC
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
GitHub user RalphSu opened a pull request:
https://github.com/apache/incubator-eagle/pull/27
EAGLE-85 : Fix unit test failure.
Fix the failed test cases.
1. avoid hive sql in test case using reserved words which fails the hive parser. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL
2. Mark not-runnable tests (like external dependency, not well formatted scala test) as ignored.
3. Fix a couple of failed test cases in hbase audit log parser.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/RalphSu/incubator-eagle master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-eagle/pull/27.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #27
----
commit f095b5688889732388bd25f7b0e80db68ff1e9a6
Author: Su <su...@gmail.com>
Date: 2015-12-14T08:34:32Z
EAGLE-85 : Fix unit test failure.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by RalphSu <gi...@git.apache.org>.
Github user RalphSu commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164414066
Updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by haoch <gi...@git.apache.org>.
Github user haoch commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164424948
https://github.com/RalphSu/incubator-eagle/blob/master/eagle-security/eagle-security-hive/src/test/java/org/apache/eagle/security/hive/ql/TestParser.java#L126
https://github.com/RalphSu/incubator-eagle/blob/master/eagle-security/eagle-security-hive/src/test/java/org/apache/eagle/security/hive/ql/TestParser.java#L153
https://github.com/RalphSu/incubator-eagle/blob/master/eagle-security/eagle-security-hive/src/test/java/org/apache/eagle/security/hive/ql/TestParser.java#L173
https://github.com/RalphSu/incubator-eagle/blob/master/eagle-security/eagle-security-hive/src/test/java/org/apache/eagle/security/hive/ql/TestParser.java#L262
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by haoch <gi...@git.apache.org>.
Github user haoch commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164463110
`shengli` should be someone's internal account name :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by haoch <gi...@git.apache.org>.
Github user haoch commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164404890
There are some sensitive table/field name in the unit test case sql, please rename/remove.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by haoch <gi...@git.apache.org>.
Github user haoch commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164467062
LGTM. Thanks very much for the contribution! @RalphSu
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-eagle/pull/27
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by haoch <gi...@git.apache.org>.
Github user haoch commented on a diff in the pull request:
https://github.com/apache/incubator-eagle/pull/27#discussion_r47482020
--- Diff: eagle-security/eagle-security-hive/src/test/java/org/apache/eagle/security/hive/ql/TestParser.java ---
@@ -248,9 +249,11 @@ public void testDropTable() throws Exception {
_testParsingQuery(query, expectedOperation, null, null);
}
+ @Ignore
@Test
public void testUnionAll() throws Exception {
- String query = "INSERT OVERWRITE TABLE be_view_event_user_t1_miss PARTITION ( dt='20151125', hour='06') select a.uid ,a.site_id ,a.page_id ,a.curprice ,a.itm ,a.itmcond ,a.itmtitle ,a.l1 ,a.l2 ,a.leaf ,a.meta ,a.st ,a.dc ,a.tr ,a.eventtimestamp ,a.cln ,a.siid ,a.ciid ,a.sellerid ,a.pri from (select a1.* from (select * from soj_view_event where dt='20151125' and hour='06') a1 inner join (select uid from soj_viewitem_event_uid_sample where dt='20151125' and hour='06') b1 on a1.uid = b1.uid) a left outer join (select c.* from (select * from be_viewitem_event_user_t1 where dt='20151125' and hour='06') a2 lateral view json_tuple(a2.values, 'u', 'site_id', 'p', 'current_price', 'item_id', 'item_condition', 'item_title', 'l1_cat_id', 'l2_cat_id', 'leaf_cat_id', 'meta_cat_id','sale_type_enum', 'shipping_country_id', 'time_remain_secs', 'timestamp', 'collection_id', 'source_impression_id', 'current_impression_id', 'sellerid' ) c as uid, site_id, page_id, curprice, itm, itmcond, i
tmtitle, l1, l2, leaf, meta, st, dc, tr, eventtimestamp, cln, siid, ciid, sellerid ) b on a.uid = b.uid and a.site_id = b.site_id and a.page_id = b.page_id and coalesce(a.curprice, 'null') = coalesce(b.curprice, 'null') and coalesce(a.itm, 'null') = coalesce(b.itm, 'null') and coalesce(a.itmcond, 'null') = coalesce(b.itmcond, 'null') and coalesce(trim(reflect(\"java.net.URLDecoder\", \"decode\",regexp_replace(a.itmtitle,\"\\\\+\",\" \"),\"utf-8\")),'null') = coalesce(trim(b.itmtitle),'null') and coalesce(a.l1, 'null') = coalesce(b.l1, 'null') and coalesce(a.l2,'null') = coalesce(b.l2,'null') and coalesce(a.leaf,'null') = coalesce(b.leaf,'null') and coalesce(a.meta,'null') = coalesce(b.meta,'null') and coalesce(a.st,'null') = coalesce(b.st,'null') and coalesce(a.dc,'null') = coalesce(b.dc,'null') and coalesce(reflect(\"java.net.URLDecoder\", \"decode\",a.tr,\"utf-8\"),'null') = coalesce(b.tr,'null') and a.eventtimestamp = b.eventtimestamp and coalesce(a.cln,'null') = coalesce(b.cln,'
null') and coalesce(a.siid,'null') = coalesce(b.siid,'null') and coalesce(a.ciid,'null') = coalesce(b.ciid,'null') and coalesce(a.sellerid, 'null') = coalesce(b.sellerid, 'null') where b.uid is null distribute by a.uid sort by a.uid , a.eventtimestamp";
+ String query = "INSERT OVERWRITE TABLE be_view_event_user_t1_miss PARTITION ( dt='20151125', hour='06') select a.uid ,a.site_id ,a.page_id ,a.curprice ,a.itm ,a.itmcond ,a.itmtitle ,a.l1 ,a.l2 ,a.leaf ,a.meta ,a.st ,a.dc ,a.tr ,a.eventtimestamp ,a.cln ,a.siid ,a.ciid ,a.sellerid ,a.pri from (select a1.* from (select * from soj_view_event where dt='20151125' and hour='06') a1 inner join (select uid from soj_viewitem_event_uid_sample where dt='20151125' and hour='06') b1 on a1.uid = b1.uid) a left outer join (select c.* from (select * from be_viewitem_event_user_t1 where dt='20151125' and hour='06') a2 lateral view json_tuple(a2.values2, 'u', 'site_id', 'p', 'current_price', 'item_id', 'item_condition', 'item_title', 'l1_cat_id', 'l2_cat_id', 'leaf_cat_id', 'meta_cat_id','sale_type_enum', 'shipping_country_id', 'time_remain_secs', 'timestamp', 'collection_id', 'source_impression_id', 'current_impression_id', 'sellerid' ) c as uid, site_id, page_id, curprice, itm, itmcond,
itmtitle, l1, l2, leaf, meta, st, dc, tr, eventtimestamp, cln, siid, ciid, sellerid ) b on a.uid = b.uid and a.site_id = b.site_id and a.page_id = b.page_id and coalesce(a.curprice, 'null') = coalesce(b.curprice, 'null') and coalesce(a.itm, 'null') = coalesce(b.itm, 'null') and coalesce(a.itmcond, 'null') = coalesce(b.itmcond, 'null') and coalesce(trim(reflect(\"java.net.URLDecoder\", \"decode\",regexp_replace(a.itmtitle,\"\\\\+\",\" \"),\"utf-8\")),'null') = coalesce(trim(b.itmtitle),'null') and coalesce(a.l1, 'null') = coalesce(b.l1, 'null') and coalesce(a.l2,'null') = coalesce(b.l2,'null') and coalesce(a.leaf,'null') = coalesce(b.leaf,'null') and coalesce(a.meta,'null') = coalesce(b.meta,'null') and coalesce(a.st,'null') = coalesce(b.st,'null') and coalesce(a.dc,'null') = coalesce(b.dc,'null') and coalesce(reflect(\"java.net.URLDecoder\", \"decode\",a.tr,\"utf-8\"),'null') = coalesce(b.tr,'null') and a.eventtimestamp = b.eventtimestamp and coalesce(a.cln,'null') = coalesce(b.cln,
'null') and coalesce(a.siid,'null') = coalesce(b.siid,'null') and coalesce(a.ciid,'null') = coalesce(b.ciid,'null') and coalesce(a.sellerid, 'null') = coalesce(b.sellerid, 'null') where b.uid is null distribute by a.uid sort by a.uid , a.eventtimestamp";
--- End diff --
Remove the sensitive table name
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by haoch <gi...@git.apache.org>.
Github user haoch commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164424435
Thanks for the fix but seems not completely clean yet :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by RalphSu <gi...@git.apache.org>.
Github user RalphSu commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164459877
Make them as more meaningless ones t1 t2....
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-eagle pull request: EAGLE-85 : Fix unit test failure.
Posted by RalphSu <gi...@git.apache.org>.
Github user RalphSu commented on the pull request:
https://github.com/apache/incubator-eagle/pull/27#issuecomment-164449409
i don't see why test_lateral_view_shengli/bm2_item4_5 are sensitive?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---