You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafu.apache.org by wang jian <ki...@gmail.com> on 2014/05/11 14:42:24 UTC
Review Request 21306: Add UDFs to handle map type data.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/
-----------------------------------------------------------
Review request for DataFu.
Bugs: https://issues.apache.org/jira/browse/DATAFU-34
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
Repository: datafu
Description
-------
Add UDFs to handle map type data.
Unit tests done.
One thing left over: validate the byte array output from map
Diffs
-----
datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java PRE-CREATION
Diff: https://reviews.apache.org/r/21306/diff/
Testing
-------
ues
Thanks,
wang jian
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by Matthew Hayes <ma...@gmail.com>.
> On May 20, 2014, 10:31 a.m., Philip (flip) Kromer wrote:
> > datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java, line 44
> > <https://reviews.apache.org/r/21306/diff/1/?file=578209#file578209line44>
> >
> > I think this would be better if the signature were PutToMap(map, kvs:bag{(key,val)}).
> >
> > That is cleaner than having to do all the odd/even checks on field counts, would let you validate the data on the front end not at execution time, and seems the most useful for how I'd think to use this:
> >
> > -- existing facts about a peep
> > peeps = LOAD 'profiles' AS (id:chararray,profile:map);
> > -- new facts about a peep. A new fact should clobber an old one with same key.
> > info = LOAD 'new_attrs' AS (id:chararray, attr:chararray, val:chararray);
> >
> > -- collect all new facts about a peep together
> > peep_info = FOREACH(GROUP info BY id) GENERATE group AS id, info.(attr,val) AS attr_vals;
> >
> > -- merge every new fact into the existing profile.
> > updated = FOREACH (JOIN peeps BY id, info BY id) GENERATE peeps::id AS id, PutToMap(profile, attr_vals) AS updated_profile;
> >
> > (written for clarity even though that only needs one COGROUP)
Good point. It would be useful to be able to operate on maps like this and merge them together. The syntax would for this example would be:
more_cars = FOREACH cars GENERATE PutToMap(price, TOMAP('bence', 1000, 'lincoln', 500));
- Matthew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/#review43466
-----------------------------------------------------------
On May 11, 2014, 12:42 p.m., wang jian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21306/
> -----------------------------------------------------------
>
> (Updated May 11, 2014, 12:42 p.m.)
>
>
> Review request for DataFu.
>
>
> Bugs: https://issues.apache.org/jira/browse/DATAFU-34
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
>
>
> Repository: datafu
>
>
> Description
> -------
>
> Add UDFs to handle map type data.
>
> Unit tests done.
>
> One thing left over: validate the byte array output from map
>
>
> Diffs
> -----
>
> datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/21306/diff/
>
>
> Testing
> -------
>
> ues
>
>
> Thanks,
>
> wang jian
>
>
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by "Philip (flip) Kromer" <fl...@infochimps.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/#review43466
-----------------------------------------------------------
datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java
<https://reviews.apache.org/r/21306/#comment77626>
The check for keep last is cheaper than looking up the key in the map, and is almost always true as it is the default case. (And it is the same every time, which CPUs love.) It might be better to structure this as
if (keep last)
put into map
else if (! has key)
put into map
else if (keep first)
do nothing
else
throw exception
datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java
<https://reviews.apache.org/r/21306/#comment77630>
I think this would be better if the signature were PutToMap(map, kvs:bag{(key,val)}).
That is cleaner than having to do all the odd/even checks on field counts, would let you validate the data on the front end not at execution time, and seems the most useful for how I'd think to use this:
-- existing facts about a peep
peeps = LOAD 'profiles' AS (id:chararray,profile:map);
-- new facts about a peep. A new fact should clobber an old one with same key.
info = LOAD 'new_attrs' AS (id:chararray, attr:chararray, val:chararray);
-- collect all new facts about a peep together
peep_info = FOREACH(GROUP info BY id) GENERATE group AS id, info.(attr,val) AS attr_vals;
-- merge every new fact into the existing profile.
updated = FOREACH (JOIN peeps BY id, info BY id) GENERATE peeps::id AS id, PutToMap(profile, attr_vals) AS updated_profile;
(written for clarity even though that only needs one COGROUP)
- Philip (flip) Kromer
On May 11, 2014, 12:42 p.m., wang jian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21306/
> -----------------------------------------------------------
>
> (Updated May 11, 2014, 12:42 p.m.)
>
>
> Review request for DataFu.
>
>
> Bugs: https://issues.apache.org/jira/browse/DATAFU-34
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
>
>
> Repository: datafu
>
>
> Description
> -------
>
> Add UDFs to handle map type data.
>
> Unit tests done.
>
> One thing left over: validate the byte array output from map
>
>
> Diffs
> -----
>
> datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/21306/diff/
>
>
> Testing
> -------
>
> ues
>
>
> Thanks,
>
> wang jian
>
>
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by Matthew Hayes <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/#review42709
-----------------------------------------------------------
Nice job on the tests :) You're covering a lot of cases. I think this is getting pretty close to being ready. Just need to look into the issues raised below.
datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java
<https://reviews.apache.org/r/21306/#comment76522>
I prefer the name ContainsAllKeys
datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java
<https://reviews.apache.org/r/21306/#comment76523>
ContainsAnyKey
datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java
<https://reviews.apache.org/r/21306/#comment76530>
rename to ContainsKeys to be consistent with proposed renames to other UDFs
datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java
<https://reviews.apache.org/r/21306/#comment76567>
I think the value type might actually be accessed like I have it below. Can you confirm this? Make sure a test case catches this if it's a problem.
input.getField(0).schema.getField(1).type
datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java
<https://reviews.apache.org/r/21306/#comment76573>
Double check whether it should be:
input.getField(0).schema.getField(1).type
datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java
<https://reviews.apache.org/r/21306/#comment76575>
Use my CheckMap UDF to verify value is an int. Same goes for other tests that produce maps.
datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java
<https://reviews.apache.org/r/21306/#comment76595>
these tuples have two fields but the bag only has val in the script
- Matthew Hayes
On May 11, 2014, 12:42 p.m., wang jian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21306/
> -----------------------------------------------------------
>
> (Updated May 11, 2014, 12:42 p.m.)
>
>
> Review request for DataFu.
>
>
> Bugs: https://issues.apache.org/jira/browse/DATAFU-34
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
>
>
> Repository: datafu
>
>
> Description
> -------
>
> Add UDFs to handle map type data.
>
> Unit tests done.
>
> One thing left over: validate the byte array output from map
>
>
> Diffs
> -----
>
> datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/21306/diff/
>
>
> Testing
> -------
>
> ues
>
>
> Thanks,
>
> wang jian
>
>
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by Will Vaughan <wi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/#review50229
-----------------------------------------------------------
For most of these, if the value of the map is a tuple with a schema, that schema ends up getting lost instead of passed alone.
Should be able to support,e.g.
(B: bag {T: tuple(key:CHARARRAY, value:tuple(v1:int, v2:CHARARRAY)})
datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java
<https://reviews.apache.org/r/21306/#comment87875>
need to handle schema
datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java
<https://reviews.apache.org/r/21306/#comment87874>
if the mapValue has a schema, that schema should be preserved in the outputSchema
- Will Vaughan
On June 7, 2014, 3:16 p.m., wang jian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21306/
> -----------------------------------------------------------
>
> (Updated June 7, 2014, 3:16 p.m.)
>
>
> Review request for DataFu.
>
>
> Bugs: https://issues.apache.org/jira/browse/DATAFU-34
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
>
>
> Repository: datafu
>
>
> Description
> -------
>
> Add UDFs to handle map type data.
>
> Unit tests done.
>
> Remaining: verify javadocs
>
>
> Diffs
> -----
>
> datafu-pig/src/main/java/datafu/pig/maps/AbstractHandleByKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/AbstractToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainsAllKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainsAnyKey.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainsKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/GetByKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapPutHandler.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/PutMapToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/RemoveByKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/SchemaValidator.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/TupleToMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/CheckBag.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/CheckMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/ContainKeysTests.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/HandleByKeysMapTests.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/ToMapTests.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/21306/diff/
>
>
> Testing
> -------
>
> ues
>
>
> Thanks,
>
> wang jian
>
>
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by wang jian <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/
-----------------------------------------------------------
(Updated June 7, 2014, 3:16 p.m.)
Review request for DataFu.
Changes
-------
(1) ./gradlew javadocs does not output generated javadocs for the new UDFs. Need to know the location of these generated docs.
(2) Add 2 new UDFs: GetByKeys and RemoveByKeys to support variable number of keys to handle from the map.
(3) Only support PutMapToMap, to put a bag to a map, use BagToMap() to convert the bag to map first.
Bugs: https://issues.apache.org/jira/browse/DATAFU-34
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
Repository: datafu
Description (updated)
-------
Add UDFs to handle map type data.
Unit tests done.
Remaining: verify javadocs
Diffs
-----
datafu-pig/src/main/java/datafu/pig/maps/AbstractHandleByKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/AbstractToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainsAllKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainsAnyKey.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainsKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/GetByKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/MapPutHandler.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/PutMapToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/RemoveByKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/SchemaValidator.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/TupleToMap.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/CheckBag.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/CheckMap.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/ContainKeysTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/HandleByKeysMapTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/ToMapTests.java PRE-CREATION
Diff: https://reviews.apache.org/r/21306/diff/
Testing
-------
ues
Thanks,
wang jian
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by wang jian <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/
-----------------------------------------------------------
(Updated June 7, 2014, 3:04 p.m.)
Review request for DataFu.
Changes
-------
Update map UDF based on code review.
Problem: I have not tested the javadocs of these new classes. Would like to know how to generate javadocs using gradlew.
i have tried ./gradlew javadoc, but does not find the output of these generated documents.
Please provide some advice on it.
Thanks
Bugs: https://issues.apache.org/jira/browse/DATAFU-34
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
Repository: datafu
Description
-------
Add UDFs to handle map type data.
Unit tests done.
One thing left over: validate the byte array output from map
Diffs (updated)
-----
datafu-pig/src/main/java/datafu/pig/maps/AbstractHandleByKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/AbstractToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainsAllKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainsAnyKey.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/ContainsKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/GetByKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/MapPutHandler.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/PutMapToMap.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/RemoveByKeys.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/SchemaValidator.java PRE-CREATION
datafu-pig/src/main/java/datafu/pig/maps/TupleToMap.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/CheckBag.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/CheckMap.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/ContainKeysTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/HandleByKeysMapTests.java PRE-CREATION
datafu-pig/src/test/java/datafu/test/pig/maps/ToMapTests.java PRE-CREATION
Diff: https://reviews.apache.org/r/21306/diff/
Testing
-------
ues
Thanks,
wang jian
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by Matthew Hayes <ma...@gmail.com>.
> On May 20, 2014, 11:04 a.m., Philip (flip) Kromer wrote:
> > datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java, line 43
> > <https://reviews.apache.org/r/21306/diff/1/?file=578205#file578205line43>
> >
> > I think here (and for ContainsAnyKey) I'd prefer it considered a bag of keys rather than an inline tuple of keys. This would let me test an arbitrary number of values rather than a fixed number.
This is a good point. Then the syntax would be like this:
has_all_cheap_cars = FILTER cars BY ContainAllKeys(price, TOBAG('rolly-royce', 'chevolet'));
This does seem cleaner to me.
One nice thing about this syntax is that you could have a bag in the relation, one that is different for each record:
has_all_cheap_cars = FILTER cars BY ContainAllKeys(price, bag_of_cars);
- Matthew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/#review43474
-----------------------------------------------------------
On May 11, 2014, 12:42 p.m., wang jian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21306/
> -----------------------------------------------------------
>
> (Updated May 11, 2014, 12:42 p.m.)
>
>
> Review request for DataFu.
>
>
> Bugs: https://issues.apache.org/jira/browse/DATAFU-34
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
>
>
> Repository: datafu
>
>
> Description
> -------
>
> Add UDFs to handle map type data.
>
> Unit tests done.
>
> One thing left over: validate the byte array output from map
>
>
> Diffs
> -----
>
> datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/21306/diff/
>
>
> Testing
> -------
>
> ues
>
>
> Thanks,
>
> wang jian
>
>
Re: Review Request 21306: Add UDFs to handle map type data.
Posted by "Philip (flip) Kromer" <fl...@infochimps.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21306/#review43474
-----------------------------------------------------------
datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java
<https://reviews.apache.org/r/21306/#comment77632>
I think here (and for ContainsAnyKey) I'd prefer it considered a bag of keys rather than an inline tuple of keys. This would let me test an arbitrary number of values rather than a fixed number.
- Philip (flip) Kromer
On May 11, 2014, 12:42 p.m., wang jian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21306/
> -----------------------------------------------------------
>
> (Updated May 11, 2014, 12:42 p.m.)
>
>
> Review request for DataFu.
>
>
> Bugs: https://issues.apache.org/jira/browse/DATAFU-34
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/DATAFU-34
>
>
> Repository: datafu
>
>
> Description
> -------
>
> Add UDFs to handle map type data.
>
> Unit tests done.
>
> One thing left over: validate the byte array output from map
>
>
> Diffs
> -----
>
> datafu-pig/src/main/java/datafu/pig/maps/BagToMap.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAllKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainAnyKey.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/ContainKeys.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/MapToBag.java PRE-CREATION
> datafu-pig/src/main/java/datafu/pig/maps/PutToMap.java PRE-CREATION
> datafu-pig/src/test/java/datafu/test/pig/maps/MapTests.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/21306/diff/
>
>
> Testing
> -------
>
> ues
>
>
> Thanks,
>
> wang jian
>
>