[jira] [Commented] (HBASE-10592) Refactor PerformanceEvaluation tool

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063162#comment-14063162 ] 

stack commented on HBASE-10592:
-------------------------------

When I try to run a MR job I get this:

{code}
4/07/15 22:02:27 INFO mapreduce.Job: Task Id : attempt_1405482830657_0004_m_000015_2, Status : FAILED
Error: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "blockEncoding" (Class org.apache.hadoop.hbase.PerformanceEvaluation$TestOptions), not marked as ignorable
 at [Source: java.io.StringReader@41c7d592; line: 1, column: 37] (through reference chain: org.apache.hadoop.hbase.TestOptions["blockEncoding"])
	at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53)
	at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:246)
	at org.codehaus.jackson.map.deser.StdDeserializer.reportUnknownProperty(StdDeserializer.java:604)
	at org.codehaus.jackson.map.deser.StdDeserializer.handleUnknownProperty(StdDeserializer.java:590)
	at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:689)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:514)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)
	at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
	at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602)
	at org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:255)
	at org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:210)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
{code}

Is I add a setter, it works.  The new JSON serialization wants setters for all properties?

> Refactor PerformanceEvaluation tool
> -----------------------------------
>
>                 Key: HBASE-10592
>                 URL: https://issues.apache.org/jira/browse/HBASE-10592
>             Project: HBase
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.96.2, 0.98.1, 0.99.0
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>            Priority: Minor
>             Fix For: 0.96.2, 0.98.1, 0.99.0, hbase-10070
>
>         Attachments: HBASE-10592.00-0.96.patch, HBASE-10592.00-0.98.patch, HBASE-10592.00.patch, HBASE-10592.01-0.96.patch, HBASE-10592.01-0.98.patch, HBASE-10592.01.patch
>
>
> PerfEval is kind of a mess. It's painful to add new features because the test options are itemized and passed as parameters to internal methods. Serialization is hand-rolled and tedious. Ensuring support for mapreduce mode is a chore because of it.
> This patch refactors the tool. Options are now passed around to methods and such as a POJO instead of one-by-one. Get rid of accessors that don't help anyone. On the mapreduce side, serialization is now handled using json (jackson is a dependency anyway) instead of the hand-rolled regex we used before. Also do away with custom InputSplit and FileFormat, instead using Text and NLineInputFormat. On the local mode side, combine 1 client and N clients into the same implementation. That implementation now uses an ExecutorService, so we can later decouple number of client workers from number of client tasks. Finally, drop a bunch of confusing local state, instead use the new TestOptions POJO as a parameter to static methods.



--
This message was sent by Atlassian JIRA
(v6.2#6252)