You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Jean-Sebastien Vachon <je...@wantedanalytics.com> on 2014/01/08 14:44:32 UTC

Migration from 0.8.3-wip to 0.9.0.1 + Netty failure

Hi All,

Yesterday I tried migrating from 0.8.3-wip to the latest release 0.9.0.1 + Netty and it fails. The topology just hangs for no apparent reason ( nothing in the worker`s or supervisor`s logs. I guess something will eventually timeout but I let it ran for about 30 minutes and no batch have completed. Each batch usually takes about 1-2 minutes to process but not 30 minutes.

I tried the same topology under ZeroMQ works fine and I got slightly better performances over 0.8.3.

The config I used for the transport is:

storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 31
storm.messaging.netty.max_wait_ms: 10000
storm.messaging.netty.min_wait_ms: 1000

This is basically the config I found on the net. I`ve only changed the max_retries after I got some timeout during the startup phase.
Any idea what could cause this? Am I the only one to run into this issue? We are still using Java 1.6 here... we had a bad experience with Java 7 and Storm a couple of months ago so I am not eager to move this project to Java 7 but will do it if helps.

Thanks for your time and input.