PATCH: fix for NPE in PostProcessorManager.java

[Kirk True <ki...@mustardgrain.com>]

Hi all,

I'm getting a NullPointerException showing up in the postprocess.log 
file because I don't have things configured quite right. Attached is a 
patch against the trunk which allows me to get past this. I'm not 
suggesting this is the correct fix :) Obviously I should configure 
things properly, but in the absence of that, we should give the user 
something instructive to help him fix the problem.

Thanks,
Kirk

Re: PATCH: fix for NPE in PostProcessorManager.java

[Ariel Rabkin <as...@gmail.com>]

Awesome. Thanks for the contribution.

Two comments:
- Can you open a JIRA for the issue?  That's our preferred way of
managing patches. (https://issues.apache.org/jira/browse/CHUKWA)
- Can you attach a copy of the stack trace you got from the NPE?
That'll help us figure out what your conf problem was, and update the
documentation (or code) accordingly.

--Ari

On Tue, Mar 16, 2010 at 4:30 PM, Kirk True <ki...@mustardgrain.com> wrote:
> Hi all,
>
> I'm getting a NullPointerException showing up in the postprocess.log file
> because I don't have things configured quite right. Attached is a patch
> against the trunk which allows me to get past this. I'm not suggesting this
> is the correct fix :) Obviously I should configure things properly, but in
> the absence of that, we should give the user something instructive to help
> him fix the problem.
>
> Thanks,
> Kirk
>
> Index:
> src/java/org/apache/hadoop/chukwa/extraction/demux/PostProcessorManager.java
> ===================================================================
> ---
> src/java/org/apache/hadoop/chukwa/extraction/demux/PostProcessorManager.java
>        (revision 924034)
> +++
> src/java/org/apache/hadoop/chukwa/extraction/demux/PostProcessorManager.java
>        (working copy)
> @@ -170,22 +170,28 @@
>   public boolean processDemuxPigOutput(String directory) throws IOException
> {
>     long start = System.currentTimeMillis();
>     try {
> -      String[] classes = conf.get(POST_DEMUX_DATA_LOADER).split(",");
> -      for(String dataLoaderName : classes) {
> -        Class<? extends DataLoaderFactory> dl = (Class<? extends
> DataLoaderFactory>) Class.forName(dataLoaderName);
> -        java.lang.reflect.Constructor<? extends DataLoaderFactory> c =
> -            dl.getConstructor();
> -        DataLoaderFactory dataloader = c.newInstance();
> +      String postDemuxDataLoaderClasses = conf.get(POST_DEMUX_DATA_LOADER);
> +
> +      if (postDemuxDataLoaderClasses != null) {
> +        String[] classes = postDemuxDataLoaderClasses.split(",");
> +        for(String dataLoaderName : classes) {
> +          Class<? extends DataLoaderFactory> dl = (Class<? extends
> DataLoaderFactory>) Class.forName(dataLoaderName);
> +          java.lang.reflect.Constructor<? extends DataLoaderFactory> c =
> +              dl.getConstructor();
> +          DataLoaderFactory dataloader = c.newInstance();
>
>           //DataLoaderFactory dataLoader = (DataLoaderFactory) Class.
>           //    forName(dataLoaderName).getConstructor().newInstance();
> -        log.info(dataLoaderName+" processing: "+directory);
> -        StringBuilder dirSearch = new StringBuilder();
> -        dirSearch.append(directory);
> -        dirSearch.append("/*/*/*.evt");
> -        Path demuxDir = new Path(dirSearch.toString());
> -        FileStatus[] events = fs.globStatus(demuxDir);
> -        dataloader.load(conf, fs, events);
> +          log.info(dataLoaderName+" processing: "+directory);
> +          StringBuilder dirSearch = new StringBuilder();
> +          dirSearch.append(directory);
> +          dirSearch.append("/*/*/*.evt");
> +          Path demuxDir = new Path(dirSearch.toString());
> +          FileStatus[] events = fs.globStatus(demuxDir);
> +          dataloader.load(conf, fs, events);
> +        }
> +      } else {
> +        log.warn(POST_DEMUX_DATA_LOADER + " value was null. Please check
> configuration");
>       }
>     } catch(Exception e) {
>       log.error(ExceptionUtil.getStackTrace(e));
>
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department