Matches on tuples

[Vincent <vi...@gmail.com>]

Good afternoon,

I am using pig on server logs to make statistics on visited pages.

For now I am able to do such matches:

- one user has visited a given page matching a given aim.
- one user has visited a given page belonging to one of the page's aim.
*
aim.id file:*

1@1@add_to_kart              --> aim 1 can be add_to_cart
1@1@browse                     --> or aim 1 can be browse
1@2@paid                         --> aim 2 is only paid

*site.log file:*

user1,1,www.site.com/browse
user1,1,www.site.com/browse
user2,1,www.site.com/add_to_kart
user2,1,www.site.com/add_to_kart
user2,1,www.site.com/paid
user2,1,www.site.com/browse
user3,1,www.site.com/browse

*Pig script:*

register 'piggybank.jar';

-- load aim id database:
aim_ids = LOAD 'aim.id' USING PigStorage('@') AS (aim_site_id : int, aim_id
: int , aim_url:chararray);

DUMP aim_ids;

-- load site log:
site = LOAD 'site.log' USING PigStorage(',') AS (user_id : chararray ,
site_id : int, url : chararray);

site_all_aims = JOIN site BY site_id, aim_ids BY aim_site_id;

site_match = FOREACH site_all_aims GENERATE user_id, site_id, aim_id,
org.apache.pig.piggybank.evaluation.string.INDEXOF(url, aim_url) AS match;

site_aims = FILTER site_match BY (match != -1) AND (match IS NOT null);

DUMP site_aims;

*results:*

(user1,1,1,13) --> user 1 achieved aim 1
(user1,1,1,13) --> user 1 achieved aim 1
(user2,1,1,13) --> user 2 achieved aim 1
(user2,1,1,13) --> user 2 achieved aim 1
(user2,1,2,13) --> user 2 achieved aim 2
(user2,1,1,13) --> user 2 achieved aim 1
(user3,1,1,13) --> user 3 achieved aim 1


Now I would like to check that a user has visited several pages to achieve
one aim. Like for a user to achieve aim 3, he needs to visit "browse" AND
"add_to_kart" AND "paid".

My idea was to load tuples of aim:

1@3@{(browse),(add_to_kart),(paid)}

And to write an UDF to compare aim URL tuple, with user's visited URL bag
for the site.

But I am not able to load tuples with an undefined number of elements. As
aims might be:

1@3@{(browse),(add_to_kart),(paid)}
1@4
@{(browse_something_else),(add_to_kart_something_else),(paid_something_else),(another_page)}

So finally, I am stuck with this problem right now, still searching for
another way to write this script and aim.id file.

If any of you as any idea, mail me.

Thanks

Vincent HERVIEUX