[GitHub] [incubator-hivemall] myui opened a new pull request #195: [HIVEMALL-259][DOC] Refactor feature_binning UDF

[GitBox <gi...@apache.org>]

myui opened a new pull request #195: [HIVEMALL-259][DOC] Refactor feature_binning UDF
URL: https://github.com/apache/incubator-hivemall/pull/195
 
 
   ## What changes were proposed in this pull request?
   
   Refactor feature_binning UDF and update the function usage
   
   ## What type of PR is it?
   
   Documentation, Refactoring
   
   ## What is the Jira issue?
   
   https://issues.apache.org/jira/browse/HIVEMALL-259
   
   ## How was this patch tested?
   
   unit tests, manual tests on EMR
   
   ## How to use this feature?
   
   ```
   WITH extracted as (
     select 
       extract_feature(feature) as index,
       extract_weight(feature) as value
     from
       input l
       LATERAL VIEW explode(features) r as feature
   ),
   mapping as (
     select
       index, 
       build_bins(value, 5, true) as quantiles -- 5 bins with auto bin shrinking
     from
       extracted
     group by
       index
   ),
   bins as (
      select 
       to_map(index, quantiles) as quantiles 
      from
       mapping
   )
   select
     l.features as original,
     feature_binning(l.features, r.quantiles) as features
   from
     input l
     cross join bins r
   ```
   
   see https://gist.github.com/myui/f943fa3ce1a7e1ac3f2dd9a7f9fa703b
   
   ## Checklist
   
   (Please remove this section if not needed; check `x` for YES, blank for NO)
   
   - [x] Did you apply source code formatter, i.e., `./bin/format_code.sh`, for your commit?
   - [x] Did you run system tests on Hive (or Spark)?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services