Date parsing Exception at Hadoop server

[Amit Dutta <am...@outlook.com>]

Hi Gurus,

 I am facing ParseException for the following code for my UDF in hive. just the evaluate method.

private final SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy", Locale.US);

public Object evaluate(DeferredObject[] arguments) throws HiveException {
    String result = "0";
    assert (arguments.length == 1);

    List<Text> list = (List<Text>) this.listOI.getList(arguments[0].get());
    if (list == null) {
        return null;
    }
    System.out.println("--------------------- Size ----------------:"+ list.get(0));
    if (list.size() > 0) {
        List<Date> listDates = new ArrayList<Date>();
        // result = compareDates(list);
        for (Text dateTxt : list) {
            try {

                        dateTxt.toString());
                String dt = new String(dateTxt.toString().trim());
                Date transDate = sdf.parse(dt);
                listDates.add(transDate);

                 listDates.size());
            } catch (ParseException e) {
                System.err.println(e.getMessage());
                e.printStackTrace();
            }
        }
        if (listDates.size() > 0) {
            Date resultDate = Collections.min(listDates);
            result = sdf.format(resultDate);
        }
    }
    return result;
}

The same code is passing the test perfectly. Following is the test method.

 public void testGetMinDate() throws HiveException {

    // set up the models we need
    GetMinDate example = new GetMinDate();
    ObjectInspector stringOI = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
    ObjectInspector listOI = ObjectInspectorFactory.getStandardListObjectInspector(stringOI);
    StringObjectInspector resultInspector = (StringObjectInspector) example.initialize(new ObjectInspector[]{listOI});

    // create the actual UDF arguments
    List<Text> list = new ArrayList<Text>();
    list.add(new Text("01-01-2015"));
    list.add(new Text("01-03-2014"));
    list.add(new Text("04-01-2015"));

    // test our results

    // the value exists
    Object result = example.evaluate(new DeferredObject[]{new DeferredJavaObject(list)});
    System.out.println(result);
    Assert.assertEquals("01-03-2014", result);

    // the value doesn't exist
//      Object result2 = example.evaluate(new DeferredObject[]{new DeferredJavaObject(list)});
//      Assert.assertEquals("Success", result2);

    // arguments are null
    Object result3 = example.evaluate(new DeferredObject[]{new DeferredJavaObject(null)});
    Assert.assertNull(result3);
  }

Following is the error

java.text.ParseException: Unparseable date: "23-05-2015"
at java.text.DateFormat.parse(DateFormat.java:357)
at com.vzw.mct.GetMinDate.evaluate(GetMinDate.java:44)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:79)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1064)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:875)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:803)
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:262)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

Not sure why it is failing in the server. If any one kindly point it out it will be great.


Thanks,

Amit