You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Rohan Kapadia <ro...@gmail.com> on 2014/03/11 09:54:53 UTC

Fwd: Unable to connect to storm drpc server

I have a storm-cluster, with everything running on the same instance
though.

I am trying to make a call from Node to the drpc server but get a
ECONNRESETerror in the js script.

On the server I get the message o.a.t.s.TNonblockingServer [ERROR] Read an
invalid frame size of -2147418111. Are you using TFramedTransport on the
client side?

*The complete error message on the client side*

making call
connection error
read ECONNRESET
Error: read ECONNRESET
    at errnoException (net.js:904:11)
    at TCP.onread (net.js:558:19)

*Client script*

var thrift = require('thrift');
var stormdrpc = require('./gen-nodejs/stormdrpc');
var ttypes = require('./gen-nodejs/test_types');
var assert = require('assert');

transport = thrift.TBufferedTransport();
protocol = thrift.TBinaryProtocol();

var connection = thrift.createConnection("10.0.1.11", 3772, {
  transport : transport,
  protocol : protocol
});

connection.on('error', function(err) {
  console.log('connection error');
  console.log(err.message);
  console.log(err.stack);
});

var client = thrift.createClient(stormdrpc, connection);

console.log("making call");
client.testdrpc('testing', function(err, response) {
  console.log("callback fired");
  console.log(err);
  console.log(response);
  console.log("callback completed");
});

*storm.yaml*

### base
storm.local.dir: "/opt/storm"
storm.local.mode.zmq: false
storm.cluster.mode: "distributed"

### zookeeper.*
storm.zookeeper.servers:
    - "10.0.2.15"
storm.zookeeper.port: 2181
storm.zookeeper.root: "/storm"
storm.zookeeper.session.timeout: 20000
storm.zookeeper.retry.times: 5
storm.zookeeper.retry.interval: 1000

### supervisor.* configs are for node supervisors
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703
supervisor.childopts: "-Xmx1024m"
supervisor.worker.start.timeout.secs: 120
supervisor.worker.timeout.secs: 30
supervisor.monitor.frequency.secs: 3
supervisor.heartbeat.frequency.secs: 5
supervisor.enable: true

### worker.* configs are for task workers
worker.childopts: "-Xmx1280m -XX:+UseConcMarkSweepGC
-Dcom.sun.management.jmxremote"
worker.heartbeat.frequency.secs: 1
task.heartbeat.frequency.secs: 3
task.refresh.poll.secs: 10
zmq.threads: 1
zmq.linger.millis: 5000

### nimbus.* configs are for the master
nimbus.host: "10.0.2.15"
nimbus.thrift.port: 6627
nimbus.childopts: "-Xmx1024m"
nimbus.task.timeout.secs: 30
nimbus.supervisor.timeout.secs: 60
nimbus.monitor.freq.secs: 10
nimbus.cleanup.inbox.freq.secs: 600
nimbus.inbox.jar.expiration.secs: 3600
nimbus.task.launch.secs: 120
nimbus.reassign: true
nimbus.file.copy.expiration.secs: 600

### ui.* configs are for the master
ui.port: 8080
ui.childopts: "-Xmx768m"

### drpc.* configs
drpc.port: 3772
drpc.worker.threads: 64
drpc.queue.size: 128
drpc.invocations.port: 3773
drpc.request.timeout.secs: 600
drpc.childopts: "-Xmx768m"
drpc.servers:
    - "10.0.2.15"

### transactional.* configs
transactional.zookeeper.servers:
    - "10.0.2.15"
transactional.zookeeper.root: "/storm-transactional"
transactional.zookeeper.port: 2181

### topology.* configs are for specific executing storms
topology.debug: true
topology.optimize: true
topology.workers: 1
topology.acker.executors: 1
topology.acker.tasks: null
topology.tasks: null
topology.message.timeout.secs: 30
topology.skip.missing.kryo.registrations: false
topology.max.task.parallelism: null
topology.max.spout.pending: null
topology.state.synchronization.timeout.secs: 60
topology.stats.sample.rate: 0.05
topology.fall.back.on.java.serialization: true
topology.worker.childopts: null




-- 
Rohan Kapadia

Re: 回复： Unable to connect to storm drpc server

Posted by Rohan Kapadia <ro...@gmail.com>.
Hi Kang,

I tried using |TframedTransport| but that also resulted in the same error?

I’m not sure what transport storm uses exactly, but from the error
message what you suggested should make sense.

Also, I am using the |TBinaryProtocol|. Is that correct, or should I use
another protocol?

I have also tried making a connection without giving a transport or
protocol, hoping that the default will work. No luck there also.

Thanks for helping out.

Regards
Rohan



On Tuesday 11 March 2014 09:07:06 PM IST, Kang Xiao wrote:
>
> Hi Rohan
>
> As the error message indicated, you should use thrift TFramedTransport
> instead of TBufferedTransport in the client script since storm drpc
> server use TFramedTransport.
>
> -- 
> Best Regards!
>
> 肖康(Kang Xiao,<kxiao.tiger@gmail.com <ma...@gmail.com>>)
> Distributed Software Engineer
>
> 在 2014年3月11日 星期二，16:54，Rohan Kapadia 写道：
>
>>
>> I have a storm-cluster, with everything running on the same instance
>> though.
>>
>> I am trying to make a call from Node to the drpc server but get a
>> |ECONNRESET| error in the js script.
>>
>> On the server I get the message |o.a.t.s.TNonblockingServer [ERROR]
>> Read an invalid frame size of -2147418111. Are you using
>> TFramedTransport on the client side?|
>>
>> *The complete error message on the client side*
>>
>> |making call
>> connection error
>> read ECONNRESET
>> Error: read ECONNRESET
>> at errnoException (net.js:904:11)
>> at TCP.onread (net.js:558:19)
>> |
>>
>> *Client script*
>>
>> |var thrift = require('thrift');
>> var stormdrpc = require('./gen-nodejs/stormdrpc');
>> var ttypes = require('./gen-nodejs/test_types');
>> var assert = require('assert');
>>
>> transport = thrift.TBufferedTransport();
>> protocol = thrift.TBinaryProtocol();
>>
>> var connection = thrift.createConnection("10.0.1.11", 3772, {
>> transport : transport,
>> protocol : protocol
>> });
>>
>> connection.on('error', function(err) {
>> console.log('connection error');
>> console.log(err.message);
>> console.log(err.stack);
>> });
>>
>> var client = thrift.createClient(stormdrpc, connection);
>>
>> console.log("making call");
>> client.testdrpc('testing', function(err, response) {
>> console.log("callback fired");
>> console.log(err);
>> console.log(response);
>> console.log("callback completed");
>> });
>> |
>>
>> *storm.yaml*
>>
>> |### base
>> storm.local.dir: "/opt/storm"
>> storm.local.mode.zmq: false
>> storm.cluster.mode: "distributed"
>>
>> ### zookeeper.*
>> storm.zookeeper.servers:
>> - "10.0.2.15"
>> storm.zookeeper.port: 2181
>> storm.zookeeper.root: "/storm"
>> storm.zookeeper.session.timeout: 20000
>> storm.zookeeper.retry.times: 5
>> storm.zookeeper.retry.interval: 1000
>>
>> ### supervisor.* configs are for node supervisors
>> supervisor.slots.ports:
>> - 6700
>> - 6701
>> - 6702
>> - 6703
>> supervisor.childopts: "-Xmx1024m"
>> supervisor.worker.start.timeout.secs: 120
>> supervisor.worker.timeout.secs: 30
>> supervisor.monitor.frequency.secs: 3
>> supervisor.heartbeat.frequency.secs: 5
>> supervisor.enable: true
>>
>> ### worker.* configs are for task workers
>> worker.childopts: "-Xmx1280m -XX:+UseConcMarkSweepGC
>> -Dcom.sun.management.jmxremote"
>> worker.heartbeat.frequency.secs: 1
>> task.heartbeat.frequency.secs: 3
>> task.refresh.poll.secs: 10
>> zmq.threads: 1
>> zmq.linger.millis: 5000
>>
>> ### nimbus.* configs are for the master
>> nimbus.host: "10.0.2.15"
>> nimbus.thrift.port: 6627
>> nimbus.childopts: "-Xmx1024m"
>> nimbus.task.timeout.secs: 30
>> nimbus.supervisor.timeout.secs: 60
>> nimbus.monitor.freq.secs: 10
>> nimbus.cleanup.inbox.freq.secs: 600
>> nimbus.inbox.jar.expiration.secs: 3600
>> nimbus.task.launch.secs: 120
>> nimbus.reassign: true
>> nimbus.file.copy.expiration.secs: 600
>>
>> ### ui.* configs are for the master
>> ui.port: 8080
>> ui.childopts: "-Xmx768m"
>>
>> ### drpc.* configs
>> drpc.port: 3772
>> drpc.worker.threads: 64
>> drpc.queue.size: 128
>> drpc.invocations.port: 3773
>> drpc.request.timeout.secs: 600
>> drpc.childopts: "-Xmx768m"
>> drpc.servers:
>> - "10.0.2.15"
>>
>> ### transactional.* configs
>> transactional.zookeeper.servers:
>> - "10.0.2.15"
>> transactional.zookeeper.root: "/storm-transactional"
>> transactional.zookeeper.port: 2181
>>
>> ### topology.* configs are for specific executing storms
>> topology.debug: true
>> topology.optimize: true
>> topology.workers: 1
>> topology.acker.executors: 1
>> topology.acker.tasks: null
>> topology.tasks: null
>> topology.message.timeout.secs: 30
>> topology.skip.missing.kryo.registrations: false
>> topology.max.task.parallelism: null
>> topology.max.spout.pending: null
>> topology.state.synchronization.timeout.secs: 60
>> topology.stats.sample.rate: 0.05
>> topology.fall.back.on.java.serialization: true
>> topology.worker.childopts: null
>> |
>>
>>
>>
>> -- 
>> Rohan Kapadia
>>
>

Re: 回复： Unable to connect to storm drpc server

Posted by Rohan Kapadia <ro...@gmail.com>.
Hi Kang,

I tried using |TframedTransport| but that also resulted in the same error?

I’m not sure what transport storm uses exactly, but from the error
message what you suggested should make sense.

Also, I am using the |TBinaryProtocol|. Is that correct, or should I use
another protocol?

I have also tried making a connection without giving a transport or
protocol, hoping that the default will work. No luck there also.

Thanks for helping out.

Regards
Rohan



On Tuesday 11 March 2014 09:07:06 PM IST, Kang Xiao wrote:
>
> Hi Rohan
>
> As the error message indicated, you should use thrift TFramedTransport
> instead of TBufferedTransport in the client script since storm drpc
> server use TFramedTransport.
>
> -- 
> Best Regards!
>
> 肖康(Kang Xiao,<kxiao.tiger@gmail.com <ma...@gmail.com>>)
> Distributed Software Engineer
>
> 在 2014年3月11日 星期二，16:54，Rohan Kapadia 写道：
>
>>
>> I have a storm-cluster, with everything running on the same instance
>> though.
>>
>> I am trying to make a call from Node to the drpc server but get a
>> |ECONNRESET| error in the js script.
>>
>> On the server I get the message |o.a.t.s.TNonblockingServer [ERROR]
>> Read an invalid frame size of -2147418111. Are you using
>> TFramedTransport on the client side?|
>>
>> *The complete error message on the client side*
>>
>> |making call
>> connection error
>> read ECONNRESET
>> Error: read ECONNRESET
>> at errnoException (net.js:904:11)
>> at TCP.onread (net.js:558:19)
>> |
>>
>> *Client script*
>>
>> |var thrift = require('thrift');
>> var stormdrpc = require('./gen-nodejs/stormdrpc');
>> var ttypes = require('./gen-nodejs/test_types');
>> var assert = require('assert');
>>
>> transport = thrift.TBufferedTransport();
>> protocol = thrift.TBinaryProtocol();
>>
>> var connection = thrift.createConnection("10.0.1.11", 3772, {
>> transport : transport,
>> protocol : protocol
>> });
>>
>> connection.on('error', function(err) {
>> console.log('connection error');
>> console.log(err.message);
>> console.log(err.stack);
>> });
>>
>> var client = thrift.createClient(stormdrpc, connection);
>>
>> console.log("making call");
>> client.testdrpc('testing', function(err, response) {
>> console.log("callback fired");
>> console.log(err);
>> console.log(response);
>> console.log("callback completed");
>> });
>> |
>>
>> *storm.yaml*
>>
>> |### base
>> storm.local.dir: "/opt/storm"
>> storm.local.mode.zmq: false
>> storm.cluster.mode: "distributed"
>>
>> ### zookeeper.*
>> storm.zookeeper.servers:
>> - "10.0.2.15"
>> storm.zookeeper.port: 2181
>> storm.zookeeper.root: "/storm"
>> storm.zookeeper.session.timeout: 20000
>> storm.zookeeper.retry.times: 5
>> storm.zookeeper.retry.interval: 1000
>>
>> ### supervisor.* configs are for node supervisors
>> supervisor.slots.ports:
>> - 6700
>> - 6701
>> - 6702
>> - 6703
>> supervisor.childopts: "-Xmx1024m"
>> supervisor.worker.start.timeout.secs: 120
>> supervisor.worker.timeout.secs: 30
>> supervisor.monitor.frequency.secs: 3
>> supervisor.heartbeat.frequency.secs: 5
>> supervisor.enable: true
>>
>> ### worker.* configs are for task workers
>> worker.childopts: "-Xmx1280m -XX:+UseConcMarkSweepGC
>> -Dcom.sun.management.jmxremote"
>> worker.heartbeat.frequency.secs: 1
>> task.heartbeat.frequency.secs: 3
>> task.refresh.poll.secs: 10
>> zmq.threads: 1
>> zmq.linger.millis: 5000
>>
>> ### nimbus.* configs are for the master
>> nimbus.host: "10.0.2.15"
>> nimbus.thrift.port: 6627
>> nimbus.childopts: "-Xmx1024m"
>> nimbus.task.timeout.secs: 30
>> nimbus.supervisor.timeout.secs: 60
>> nimbus.monitor.freq.secs: 10
>> nimbus.cleanup.inbox.freq.secs: 600
>> nimbus.inbox.jar.expiration.secs: 3600
>> nimbus.task.launch.secs: 120
>> nimbus.reassign: true
>> nimbus.file.copy.expiration.secs: 600
>>
>> ### ui.* configs are for the master
>> ui.port: 8080
>> ui.childopts: "-Xmx768m"
>>
>> ### drpc.* configs
>> drpc.port: 3772
>> drpc.worker.threads: 64
>> drpc.queue.size: 128
>> drpc.invocations.port: 3773
>> drpc.request.timeout.secs: 600
>> drpc.childopts: "-Xmx768m"
>> drpc.servers:
>> - "10.0.2.15"
>>
>> ### transactional.* configs
>> transactional.zookeeper.servers:
>> - "10.0.2.15"
>> transactional.zookeeper.root: "/storm-transactional"
>> transactional.zookeeper.port: 2181
>>
>> ### topology.* configs are for specific executing storms
>> topology.debug: true
>> topology.optimize: true
>> topology.workers: 1
>> topology.acker.executors: 1
>> topology.acker.tasks: null
>> topology.tasks: null
>> topology.message.timeout.secs: 30
>> topology.skip.missing.kryo.registrations: false
>> topology.max.task.parallelism: null
>> topology.max.spout.pending: null
>> topology.state.synchronization.timeout.secs: 60
>> topology.stats.sample.rate: 0.05
>> topology.fall.back.on.java.serialization: true
>> topology.worker.childopts: null
>> |
>>
>>
>>
>> -- 
>> Rohan Kapadia
>>
>

回复： Unable to connect to storm drpc server

Posted by Kang Xiao <kx...@gmail.com>.
Hi Rohan  

As the error message indicated, you should use thrift TFramedTransport instead of TBufferedTransport in the client script since storm drpc server use TFramedTransport.  

--  
Best Regards!

肖康(Kang Xiao,<kxiao.tiger@gmail.com (mailto:kxiao.tiger@gmail.com)>)
Distributed Software Engineer


在 2014年3月11日 星期二，16:54，Rohan Kapadia 写道：

>  
> I have a storm-cluster, with everything running on the same instance though.  
>  
>  
> I am trying to make a call from Node to the drpc server but get a ECONNRESET error in the js script.  
>  
>  
> On the server I get the message o.a.t.s.TNonblockingServer [ERROR] Read an invalid frame size of -2147418111. Are you using TFramedTransport on the client side?
>  
>  
> The complete error message on the client side
>  
> making call connection error read ECONNRESET Error: read ECONNRESET at errnoException (net.js:904:11) at TCP.onread (net.js:558:19)  
>  
> Client script
>  
> var thrift = require('thrift'); var stormdrpc = require('./gen-nodejs/stormdrpc'); var ttypes = require('./gen-nodejs/test_types'); var assert = require('assert'); transport = thrift.TBufferedTransport(); protocol = thrift.TBinaryProtocol(); var connection = thrift.createConnection("10.0.1.11", 3772, { transport : transport, protocol : protocol }); connection.on('error', function(err) { console.log('connection error'); console.log(err.message); console.log(err.stack); }); var client = thrift.createClient(stormdrpc, connection); console.log("making call"); client.testdrpc('testing', function(err, response) { console.log("callback fired"); console.log(err); console.log(response); console.log("callback completed"); });  
>  
> storm.yaml
>  
> ### base storm.local.dir: "/opt/storm" storm.local.mode.zmq: false storm.cluster.mode: "distributed" ### zookeeper.* storm.zookeeper.servers: - "10.0.2.15" storm.zookeeper.port: 2181 storm.zookeeper.root: "/storm" storm.zookeeper.session.timeout: 20000 storm.zookeeper.retry.times: 5 storm.zookeeper.retry.interval: 1000 ### supervisor.* configs are for node supervisors supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703 supervisor.childopts: "-Xmx1024m" supervisor.worker.start.timeout.secs: 120 supervisor.worker.timeout.secs: 30 supervisor.monitor.frequency.secs: 3 supervisor.heartbeat.frequency.secs: 5 supervisor.enable: true ### worker.* configs are for task workers worker.childopts: "-Xmx1280m -XX:+UseConcMarkSweepGC -Dcom.sun.management.jmxremote" worker.heartbeat.frequency.secs: 1 task.heartbeat.frequency.secs: 3 task.refresh.poll.secs: 10 zmq.threads: 1 zmq.linger.millis: 5000 ### nimbus.* configs are for the master nimbus.host: "10.0.2.15" nimbus.thrift.port: 6627 nimbus.childopts: "-Xmx1024m" nimbus.task.timeout.secs: 30 nimbus.supervisor.timeout.secs: 60 nimbus.monitor.freq.secs: 10 nimbus.cleanup.inbox.freq.secs: 600 nimbus.inbox.jar.expiration.secs: 3600 nimbus.task.launch.secs: 120 nimbus.reassign: true nimbus.file.copy.expiration.secs: 600 ### ui.* configs are for the master ui.port: 8080 ui.childopts: "-Xmx768m" ### drpc.* configs drpc.port: 3772 drpc.worker.threads: 64 drpc.queue.size: 128 drpc.invocations.port: 3773 drpc.request.timeout.secs: 600 drpc.childopts: "-Xmx768m" drpc.servers: - "10.0.2.15" ### transactional.* configs transactional.zookeeper.servers: - "10.0.2.15" transactional.zookeeper.root: "/storm-transactional" transactional.zookeeper.port: 2181 ### topology.* configs are for specific executing storms topology.debug: true topology.optimize: true topology.workers: 1 topology.acker.executors: 1 topology.acker.tasks: null topology.tasks: null topology.message.timeout.secs: 30 topology.skip.missing.kryo.registrations: false topology.max.task.parallelism: null topology.max.spout.pending: null topology.state.synchronization.timeout.secs: 60 topology.stats.sample.rate: 0.05 topology.fall.back.on.java.serialization: true topology.worker.childopts: null  
>  
>  
>  
>  
>  
>  
> --  
> Rohan Kapadia
>