You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by researcher cs <pr...@gmail.com> on 2016/01/27 01:29:36 UTC
DRPC server not working
I set in the code
conf.put(Config.DRPC_SERVERS, dprcServers);
conf.put(Config.DRPC_PORT, 3772);
but when i submit topolgoy i found at the end of the file
b.s.d.drpc [INFO] Starting Distributed RPC servers...
i checked port 3772 by
sudo netstat -ap | grep 3772
i got
tcp 0 0 *:3772 *:* LISTEN 10864/java
unix 3 [ ] STREAM CONNECTED 13772
587/dbus-daemon /var/run/dbus/system_bus_socket
why it's not established ?
can i find help ?
Re: DRPC server not working
Posted by sam mohel <sa...@gmail.com>.
Sorry I sent from another email
On Wednesday, January 27, 2016, sam mohel <sa...@gmail.com> wrote:
> I wrote the actual problem in My first message drpc server not working I
> hot zeros in the columns of storm ui like emitted and transferred , result
> file is empty
>
> On Wednesday, January 27, 2016, Erik Weathers
> <ew...@groupon.com.invalid> wrote:
>
>> Your mail client is wrapping the log lines prematurely, I have a really
>> really hard time reading wrapped lines, I'd look into fixing that if I
>> were
>> you. Here they are unwrapped:
>>
>> 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
>> connectString=localhost:2181 sessionTimeout=20000
>> watcher=com.netflix.curator.ConnectionState@2fa423d2
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
>> server localhost/127.0.1.1:2181. Will not attempt to authenticate using
>> SASL (unknown error)
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
>> to localhost/127.0.1.1:2181, initiating session
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
>> on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
>> negotiated timeout = 20000
>> 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
>> :connected:none
>> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
>> closed
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
>> 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
>> connectString=localhost:2181/storm sessionTimeout=20000
>> watcher=com.netflix.curator.ConnectionState@2ee8b0bf
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
>> server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
>> authenticate
>> using SASL (unknown error)
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
>> to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
>> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
>> on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
>> 0x152804f3a3a0003,
>> negotiated timeout = 20000
>>
>> None of those indicate a problem, they look pretty standard to me.
>>
>> Please spend a bit more time zeroing in on what the actual problem is so
>> that the members of the list(s) can provide help.
>>
>> - Erik
>>
>> On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
>> prog.researcher@gmail.com>
>> wrote:
>>
>> > yes i tried in local and worked well
>> > and about /etc/hosts . i'm feeling that this file has a mistake , i made
>> > alot of changes in this file and didn't remember what was default
>> >
>> > In nimbus log file when it connected zookeeper i got
>> > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
>> > connectString=localhost:2181 sessionTimeout=20000
>> > watcher=com.netflix.curator.ConnectionState@2fa423d2
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
>> > server localhost/127.0.1.1:2181. Will not attempt to authenticate using
>> > SASL (unknown error)
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>> established
>> > to localhost/127.0.1.1:2181, initiating session
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>> complete
>> > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
>> > negotiated timeout = 20000
>> > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
>> > :connected:none
>> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
>> > closed
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
>> > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
>> > connectString=localhost:2181/storm sessionTimeout=20000
>> > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
>> > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
>> authenticate
>> > using SASL (unknown error)
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>> established
>> > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
>> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>> complete
>> > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
>> > 0x152804f3a3a0003,
>> > negotiated timeout = 20000
>> >
>> > and i set for drpc.server : localhost
>> > storm.zookeeper.server : localhost
>> > nimbus.host : localhost
>> >
>> > in my /etc/hosts
>> > 127.0.0.1 user-Lenovo-G50-70 localhost
>> > 127.0.1.1 localhost
>> >
>> > is that right ?
>> >
>> >
>> > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
>> > eweathers@groupon.com.invalid> wrote:
>> >
>> > > You said: "except the statement of drpc server trying to connect"
>> > >
>> > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
>> Distributed
>> > > RPC servers..." implies?
>> > > That is just saying that the server is being started. It's a server,
>> > not
>> > > a client, so it's basic operation is *not* to connect to some other
>> > thing.
>> > > It's up and waiting for you to tell it to do stuff.
>> > >
>> > > Have you gotten the Local-mode version of DRPC working?
>> > >
>> > > -
>> > >
>> > >
>> >
>> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
>> > >
>> > > Not sure what you're asking with regards to DNS and /etc/hosts, those
>> > seem
>> > > unrelated to your basic issue.
>> > >
>> > > - Erik
>> > >
>> > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
>> > prog.researcher@gmail.com>
>> > > wrote:
>> > >
>> > > > There is no error except the statement of drpc server trying to
>> connect
>> > > . I
>> > > > guess I have problem with Dns . if you have any idea about this
>> pleaes
>> > > help
>> > > >
>> > > >
>> > > > I want to submit topology with single machine
>> > > > Only on my laptop without any other devices as a first step so
>> > > > What this file /etc/hosts should contains ?
>> > > > As I set in drpc.server : localhost
>> > > > Storm.zookeeper. server: localhost
>> > > > Nimbus.host:localhost
>> > > >
>> > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
>> > > >
>> > > > What should I use and what should I hash it to not using it ?
>> > > >
>> > > >
>> > > >
>> > > > On Wednesday, January 27, 2016, Erik Weathers
>> > > > <ew...@groupon.com.invalid>
>> > > > wrote:
>> > > >
>> > > > > What does the client code that is supposed to make the DRPC
>> > connection
>> > > > > telling you? i.e., you should see some exception or log about not
>> > > > > establishing the connection, right?
>> > > > >
>> > > > > Alternatively, perhaps the connections aren't persistent and
>> there's
>> > no
>> > > > > actual problem?
>> > > > >
>> > > > > - Erik
>> > > > >
>> > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
>> > > > prog.researcher@gmail.com
>> > > > > <javascript:;>>
>> > > > > wrote:
>> > > > >
>> > > > > > thanks for replying , i read the documentation before , i
>> imported
>> > > > > project
>> > > > > > supposed to work well but not working with me
>> > > > > > i checked port by lsof -i gave me all ports i connected it for
>> > storm
>> > > > > > java 10675 root 20u IPv4 98126 0t0 TCP *:52022
>> > > > (LISTEN)
>> > > > > > java 10675 root 26u IPv4 98131 0t0 TCP *:2181
>> > > (LISTEN)
>> > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
>> > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
>> > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
>> > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
>> > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
>> > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
>> > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
>> > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
>> > > > > > java 10715 root 91u IPv4 98245 0t0 TCP *:6627
>> > > (LISTEN)
>> > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
>> > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
>> > > > > > java 10864 root 82u IPv4 102425 0t0 TCP *:3772
>> > > (LISTEN)
>> > > > > > java 10864 root 84u IPv4 102429 0t0 TCP *:3773
>> > > (LISTEN)
>> > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
>> > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
>> > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
>> *:http-alt
>> > > > > (LISTEN)
>> > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
>> > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
>> > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
>> > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
>> > > > > > java 11087 root 94u IPv4 102561 0t0 TCP *:7660
>> > > (LISTEN)
>> > > > > >
>> > > > > >
>> > > > > > here you can see that 3772 not established
>> > > > > >
>> > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
>> > > > > > eweathers@groupon.com.invalid> wrote:
>> > > > > >
>> > > > > > > hey,
>> > > > > > >
>> > > > > > > The DRPC server is up and listening on port 3772. Why do you
>> > > expect
>> > > > > > > established connections?
>> > > > > > >
>> > > > > > > I'm not familiar with using Storm's DRPC feature, but I'm sure
>> > you
>> > > > need
>> > > > > > to
>> > > > > > > write code that interacts with the DRPC server, and you've
>> made
>> > no
>> > > > > > mention
>> > > > > > > of doing so in your email. I'd start here:
>> > > > > > >
>> > > > > > > -
>> https://storm.apache.org/documentation/Distributed-RPC.html
>> > > > > > >
>> > > > > > > - Erik
>> > > > > > >
>> > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
>> > > > > > prog.researcher@gmail.com <javascript:;>>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > I set in the code
>> > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
>> > > > > > > > conf.put(Config.DRPC_PORT, 3772);
>> > > > > > > > but when i submit topolgoy i found at the end of the file
>> > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
>> > > > > > > >
>> > > > > > > > i checked port 3772 by
>> > > > > > > >
>> > > > > > > > sudo netstat -ap | grep 3772
>> > > > > > > >
>> > > > > > > > i got
>> > > > > > > >
>> > > > > > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
>> > > > > > > > unix 3 [ ] STREAM CONNECTED 13772
>> > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > why it's not established ?
>> > > > > > > >
>> > > > > > > > can i find help ?
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
Is there anybody here used dtpc before ?
On Thursday, January 28, 2016, researcher cs <pr...@gmail.com>
wrote:
> storm vresion -0.9.0.1 zookeeper -3.4.6 i tried example of nathan marz but
> didn't work too , gave me same result so i felt it's my problem with
> configuration
> as i'm using one machine for distributed but someone told me that i should
> first submit topology with single machine wit ip 127.0.0.1 first to make
> sure it's working well in distrbuted then i can make real network
>
> On Thu, Jan 28, 2016 at 1:56 AM, Erik Weathers <
> eweathers@groupon.com.invalid
> <javascript:_e(%7B%7D,'cvml','eweathers@groupon.com.invalid');>> wrote:
>
>> Not sure I'll have time, but if I do then I'll try to get the DRPC starter
>> example working and then can probably tell you more explicitly how to make
>> your example work.
>>
>> Can you please tell us which version of Storm you are using?
>>
>> - Erik
>>
>> On Wed, Jan 27, 2016 at 3:25 PM, researcher cs <prog.researcher@gmail.com
>> <javascript:_e(%7B%7D,'cvml','prog.researcher@gmail.com');>>
>> wrote:
>>
>> > can you try this project ? i hope you can
>> >
>>
>
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
storm vresion -0.9.0.1 zookeeper -3.4.6 i tried example of nathan marz but
didn't work too , gave me same result so i felt it's my problem with
configuration
as i'm using one machine for distributed but someone told me that i should
first submit topology with single machine wit ip 127.0.0.1 first to make
sure it's working well in distrbuted then i can make real network
On Thu, Jan 28, 2016 at 1:56 AM, Erik Weathers <
eweathers@groupon.com.invalid> wrote:
> Not sure I'll have time, but if I do then I'll try to get the DRPC starter
> example working and then can probably tell you more explicitly how to make
> your example work.
>
> Can you please tell us which version of Storm you are using?
>
> - Erik
>
> On Wed, Jan 27, 2016 at 3:25 PM, researcher cs <pr...@gmail.com>
> wrote:
>
> > can you try this project ? i hope you can
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
Not sure I'll have time, but if I do then I'll try to get the DRPC starter
example working and then can probably tell you more explicitly how to make
your example work.
Can you please tell us which version of Storm you are using?
- Erik
On Wed, Jan 27, 2016 at 3:25 PM, researcher cs <pr...@gmail.com>
wrote:
> can you try this project ? i hope you can
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
can you try this project ? i hope you can
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
I really don't think you can have any expectation of this doing anything
without somehow shoving work into it. Look at all the things being done in
the "local cluster" block versus the "remote" block of your code.
Would be nice if someone who has ever used DRPC could respond. The docs
seem to be lacking in this critical area of getting it to work with a
remote cluster. i.e., the 3 steps definitely don't seem to be sufficient.
- Erik
On Wed, Jan 27, 2016 at 3:16 PM, researcher cs <pr...@gmail.com>
wrote:
> thanks but Agian i imported this project from github that it's supposed
> work well , really i'm on this problem for months without fixing it !! i
> sent coder message but he didn't reply with this problem . don't know who
> can i ask but thanks for your time
>
> On Thu, Jan 28, 2016 at 1:12 AM, Erik Weathers <
> eweathers@groupon.com.invalid> wrote:
>
> > hey Sam,
> >
> > Again, I've never used the DRPC feature. But it naively looks like you
> > aren't using it correctly.
> > Note that the working case for you is with a LocalCluster & LocalDRPC,
> and
> > you are explicitly invoking the execute():
> > String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
> >
> > In the remote case you aren't doing anything except submitting the
> > topology. I assume you need to invoke drpc.execute *somewhere*... Here's
> > what I think is the same question as you are posing:
> >
> > - http://stackoverflow.com/a/26440260/318428
> >
> > - Erik
> >
> > On Wed, Jan 27, 2016 at 3:04 PM, researcher cs <
> prog.researcher@gmail.com>
> > wrote:
> >
> > > also got zeros after open storm ui except executed column
> > >
> > >
> > >
> > >
> > > On Thu, Jan 28, 2016 at 12:43 AM, researcher cs <
> > prog.researcher@gmail.com
> > > > wrote:
> > >
> > >> i atteached what i got when i submitted topology and this happened
> also
> > >> when i submitted
> storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> > >> storm.starter.BasicDRPCTopology basic-drpc
> > >>
> > >> this storm.yaml
> > >> storm.zookeeper.servers:
> > >> - "localhost"
> > >> # - "server2"
> > >> nimbus.host: "localhost"
> > >>
> > >> storm.local.dir: "/var/storm"
> > >> supervisor.slots.ports:
> > >> - 7660
> > >> - 7659
> > >> - 7658
> > >> - 7657
> > >> supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
> > >> nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
> > >> worker.childopts: "-Djava.net.preferIPv4Stack=true"
> > >> # topology.message.timeout.secs: 30
> > >> # topology.workers: 1
> > >> topology.stats.sample.rate: 1.0
> > >> topology.acker.executors: 1
> > >> # topology.executor.receive.buffer.size: 16384
> > >> # topology.executor.send.buffer.size: 16384
> > >> # topology.transfer.buffer.size: 32
> > >> # topology.receiver.buffer.size: 8
> > >> #
> > >> # ##### These may optionally be filled in:
> > >> #
> > >> ## List of custom serializations
> > >> # topology.kryo.register:
> > >> # - org.mycompany.MyType
> > >> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer
> > >> #
> > >> ## List of custom kryo decorators
> > >> # topology.kryo.decorators:
> > >> # - org.mycompany.MyDecorator
> > >> #
> > >> ## Locations of the drpc servers
> > >> # drpc.servers:
> > >> # - "localhost"
> > >> # - "server2"
> > >>
> > >> ## Metrics Consumers
> > >> # topology.metrics.consumer.register:
> > >> # - class: "backtype.storm.metrics.LoggingMetricsConsumer"
> > >> # parallelism.hint: 1
> > >> # - class: "org.mycompany.MyMetricsConsumer"
> > >> # parallelism.hint: 1
> > >> # argument:
> > >> # - endpoint: "metrics-collector.mycompany.org"
> > >> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
> > >> storm.messaging.netty.server_worker_threads: 1
> > >> storm.messaging.netty.client_worker_threads: 1
> > >> storm.messaging.netty.buffer_size: 5242880
> > >> storm.messaging.netty.max_retries: 100
> > >> storm.messaging.netty.max_wait_ms: 1000
> > >> storm.messaging.netty.min_wait_ms: 100
> > >>
> > >> this etc/hosts
> > >> 127.0.0.1 localhost
> > >> 127.0.1.1 user-Lenovo-G50-70
> > >> ##The following lines are desirable for IPv6 capable hosts##
> > >> ::1 ip6-localhost ip6-loopback
> > >> fe00::0 ip6-localnet
> > >> ff00::0 ip6-mcastprefix
> > >> ff02::1 ip6-allnodes
> > >> ff02::2 ip6-allrouters
> > >>
> > >> i set drpc in code
> > >> private static Config createTopologyConfiguration(Properties prop,
> > >> boolean localMode) {
> > >> Config conf = new Config();
> > >> List<String> dprcServers = new ArrayList<String>();
> > >> dprcServers.add("localhost");
> > >>
> > >> conf.put(Config.DRPC_SERVERS, dprcServers);
> > >> conf.put(Config.DRPC_PORT, 3772);
> > >> if (!localMode)
> > >> conf.put(Config.STORM_CLUSTER_MODE, new
> > >> String("distributed"));
> > >>
> > >> }
> > >> public static void main(String[] args) throws Exception {
> > >> if (args == null || args.length == 0) {
> > >> Config conf = createTopologyConfiguration(prop, true);
> > >> LocalDRPC drpc = new LocalDRPC();
> > >> LocalCluster cluster = new LocalCluster();
> > >>
> > >> cluster.submitTopology(TOPOLOGY_NAME, conf,
> > >> buildTopology(drpc));
> > >>
> > >> Thread.sleep(2000); // give it some time to setup
> > >>
> > >> BufferedReader br = new BufferedReader(new
> > >> FileReader(queryFile));
> > >> String tweetJson;
> > >> fos.write("Start: ".getBytes());
> > >>
> > >> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
> > >> byte[] newLine = "\n".getBytes();
> > >> int times = 0;
> > >> // emit tweets into topology
> > >> while ((tweetJson = br.readLine()) != null) {
> > >>
> > >> String result = drpc.execute(TOPOLOGY_NAME,
> tweetJson);
> > >>
> > >> Status s = null;
> > >> try {
> > >> s = DataObjectFactory.createStatus(tweetJson);
> > >> result = s.getId() + "\t" + s.getText() + "\t" +
> > >> result;
> > >> } catch (TwitterException e) {
> > >> LOG.error(e.toString());
> > >> }
> > >>
> > >> fos.write(result.getBytes());
> > >> fos.write(newLine);
> > >>
> > >> // times++;
> > >> // if (times == 1000)
> > >> // break;
> > >> }
> > >> fos.write(newLine);
> > >> fos.write("Finish: ".getBytes());
> > >>
> > >> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
> > >>
> > >> fos.flush();
> > >> fos.close();
> > >> br.close();
> > >> drpc.shutdown();
> > >> cluster.shutdown();
> > >> } else {
> > >> // distributed mode
> > >> Config conf = createTopologyConfiguration(prop,true);
> > >> LocalDRPC drpc = null;
> > >> StormSubmitter.submitTopology(args[0], conf,
> > >> buildTopology(drpc));
> > >>
> > >> }
> > >>
> > >> }
> > >>
> > >> On Wed, Jan 27, 2016 at 11:14 PM, Erik Weathers <
> > >> eweathers@groupon.com.invalid> wrote:
> > >>
> > >>> Please put more effort into describing the issue. "It doesn't work"
> is
> > >>> unfortunately not enough info for anyone to provide help.
> > >>> e.g., post links to some code you are trying to run, and the configs
> of
> > >>> the
> > >>> storm components that you are running.
> > >>>
> > >>> - Erik
> > >>>
> > >>> On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sa...@gmail.com>
> > wrote:
> > >>>
> > >>> > I wrote the actual problem in My first message drpc server not
> > working
> > >>> I
> > >>> > hot zeros in the columns of storm ui like emitted and transferred ,
> > >>> result
> > >>> > file is empty
> > >>> >
> > >>> > On Wednesday, January 27, 2016, Erik Weathers
> > >>> > <ew...@groupon.com.invalid>
> > >>> > wrote:
> > >>> >
> > >>> > > Your mail client is wrapping the log lines prematurely, I have a
> > >>> really
> > >>> > > really hard time reading wrapped lines, I'd look into fixing that
> > if
> > >>> I
> > >>> > were
> > >>> > > you. Here they are unwrapped:
> > >>> > >
> > >>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO]
> Starting
> > >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> > >>> connection,
> > >>> > > connectString=localhost:2181 sessionTimeout=20000
> > >>> > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> > >>> connection to
> > >>> > > server localhost/127.0.1.1:2181. Will not attempt to
> authenticate
> > >>> using
> > >>> > > SASL (unknown error)
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> > >>> established
> > >>> > > to localhost/127.0.1.1:2181, initiating session
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > >>> > complete
> > >>> > > on server localhost/127.0.1.1:2181, sessionid =
> 0x152804f3a3a0002,
> > >>> > > negotiated timeout = 20000
> > >>> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> > >>> > > :connected:none
> > >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session:
> > 0x152804f3a3a0002
> > >>> > > closed
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> > >>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO]
> Starting
> > >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> > >>> connection,
> > >>> > > connectString=localhost:2181/storm sessionTimeout=20000
> > >>> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> > >>> connection to
> > >>> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> > >>> > authenticate
> > >>> > > using SASL (unknown error)
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> > >>> established
> > >>> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > >>> > complete
> > >>> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > >>> > > 0x152804f3a3a0003,
> > >>> > > negotiated timeout = 20000
> > >>> > >
> > >>> > > None of those indicate a problem, they look pretty standard to
> me.
> > >>> > >
> > >>> > > Please spend a bit more time zeroing in on what the actual
> problem
> > >>> is so
> > >>> > > that the members of the list(s) can provide help.
> > >>> > >
> > >>> > > - Erik
> > >>> > >
> > >>> > > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
> > >>> > prog.researcher@gmail.com
> > >>> > > <javascript:;>>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > yes i tried in local and worked well
> > >>> > > > and about /etc/hosts . i'm feeling that this file has a mistake
> > , i
> > >>> > made
> > >>> > > > alot of changes in this file and didn't remember what was
> default
> > >>> > > >
> > >>> > > > In nimbus log file when it connected zookeeper i got
> > >>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO]
> > Starting
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> > >>> > connection,
> > >>> > > > connectString=localhost:2181 sessionTimeout=20000
> > >>> > > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> > >>> connection
> > >>> > to
> > >>> > > > server localhost/127.0.1.1:2181. Will not attempt to
> > authenticate
> > >>> > using
> > >>> > > > SASL (unknown error)
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> > >>> > established
> > >>> > > > to localhost/127.0.1.1:2181, initiating session
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session
> establishment
> > >>> > > complete
> > >>> > > > on server localhost/127.0.1.1:2181, sessionid =
> > 0x152804f3a3a0002,
> > >>> > > > negotiated timeout = 20000
> > >>> > > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state
> update:
> > >>> > > > :connected:none
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session:
> > >>> 0x152804f3a3a0002
> > >>> > > > closed
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut
> down
> > >>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO]
> > Starting
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> > >>> > connection,
> > >>> > > > connectString=localhost:2181/storm sessionTimeout=20000
> > >>> > > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> > >>> connection
> > >>> > to
> > >>> > > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> > >>> > > authenticate
> > >>> > > > using SASL (unknown error)
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> > >>> > established
> > >>> > > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session
> establishment
> > >>> > > complete
> > >>> > > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > >>> > > > 0x152804f3a3a0003,
> > >>> > > > negotiated timeout = 20000
> > >>> > > >
> > >>> > > > and i set for drpc.server : localhost
> > >>> > > > storm.zookeeper.server : localhost
> > >>> > > > nimbus.host : localhost
> > >>> > > >
> > >>> > > > in my /etc/hosts
> > >>> > > > 127.0.0.1 user-Lenovo-G50-70 localhost
> > >>> > > > 127.0.1.1 localhost
> > >>> > > >
> > >>> > > > is that right ?
> > >>> > > >
> > >>> > > >
> > >>> > > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
> > >>> > > > eweathers@groupon.com.invalid> wrote:
> > >>> > > >
> > >>> > > > > You said: "except the statement of drpc server trying to
> > connect"
> > >>> > > > >
> > >>> > > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
> > >>> > > Distributed
> > >>> > > > > RPC servers..." implies?
> > >>> > > > > That is just saying that the server is being started. It's
> a
> > >>> > server,
> > >>> > > > not
> > >>> > > > > a client, so it's basic operation is *not* to connect to some
> > >>> other
> > >>> > > > thing.
> > >>> > > > > It's up and waiting for you to tell it to do stuff.
> > >>> > > > >
> > >>> > > > > Have you gotten the Local-mode version of DRPC working?
> > >>> > > > >
> > >>> > > > > -
> > >>> > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
> > >>> > > > >
> > >>> > > > > Not sure what you're asking with regards to DNS and
> /etc/hosts,
> > >>> those
> > >>> > > > seem
> > >>> > > > > unrelated to your basic issue.
> > >>> > > > >
> > >>> > > > > - Erik
> > >>> > > > >
> > >>> > > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
> > >>> > > > prog.researcher@gmail.com <javascript:;>>
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > There is no error except the statement of drpc server
> trying
> > to
> > >>> > > connect
> > >>> > > > > . I
> > >>> > > > > > guess I have problem with Dns . if you have any idea about
> > this
> > >>> > > pleaes
> > >>> > > > > help
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > I want to submit topology with single machine
> > >>> > > > > > Only on my laptop without any other devices as a first step
> > so
> > >>> > > > > > What this file /etc/hosts should contains ?
> > >>> > > > > > As I set in drpc.server : localhost
> > >>> > > > > > Storm.zookeeper. server: localhost
> > >>> > > > > > Nimbus.host:localhost
> > >>> > > > > >
> > >>> > > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP
> > >>> address ?
> > >>> > > > > >
> > >>> > > > > > What should I use and what should I hash it to not using
> it ?
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > On Wednesday, January 27, 2016, Erik Weathers
> > >>> > > > > > <ew...@groupon.com.invalid>
> > >>> > > > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > What does the client code that is supposed to make the
> DRPC
> > >>> > > > connection
> > >>> > > > > > > telling you? i.e., you should see some exception or log
> > >>> about
> > >>> > not
> > >>> > > > > > > establishing the connection, right?
> > >>> > > > > > >
> > >>> > > > > > > Alternatively, perhaps the connections aren't persistent
> > and
> > >>> > > there's
> > >>> > > > no
> > >>> > > > > > > actual problem?
> > >>> > > > > > >
> > >>> > > > > > > - Erik
> > >>> > > > > > >
> > >>> > > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > >>> > > > > > prog.researcher@gmail.com <javascript:;>
> > >>> > > > > > > <javascript:;>>
> > >>> > > > > > > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > thanks for replying , i read the documentation before
> ,
> > i
> > >>> > > imported
> > >>> > > > > > > project
> > >>> > > > > > > > supposed to work well but not working with me
> > >>> > > > > > > > i checked port by lsof -i gave me all ports i connected
> > it
> > >>> for
> > >>> > > > storm
> > >>> > > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP
> > >>> *:52022
> > >>> > > > > > (LISTEN)
> > >>> > > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP
> > >>> *:2181
> > >>> > > > > (LISTEN)
> > >>> > > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
> > >>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > >>> > > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
> > >>> > > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526
> > >>> (ESTABLISHED)
> > >>> > > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
> > >>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > >>> > > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
> > >>> > > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > >>> > > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP
> > >>> *:6627
> > >>> > > > > (LISTEN)
> > >>> > > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
> > >>> > > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181
> > >>> (ESTABLISHED)
> > >>> > > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP
> > >>> *:3772
> > >>> > > > > (LISTEN)
> > >>> > > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP
> > >>> *:3773
> > >>> > > > > (LISTEN)
> > >>> > > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
> > >>> > > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825
> > >>> (ESTABLISHED)
> > >>> > > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
> > >>> > > *:http-alt
> > >>> > > > > > > (LISTEN)
> > >>> > > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
> > >>> > > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773
> > >>> (ESTABLISHED)
> > >>> > > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
> > >>> > > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > >>> > > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP
> > >>> *:7660
> > >>> > > > > (LISTEN)
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > > here you can see that 3772 not established
> > >>> > > > > > > >
> > >>> > > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > >>> > > > > > > > eweathers@groupon.com.invalid> wrote:
> > >>> > > > > > > >
> > >>> > > > > > > > > hey,
> > >>> > > > > > > > >
> > >>> > > > > > > > > The DRPC server is up and listening on port 3772.
> Why
> > >>> do
> > >>> > you
> > >>> > > > > expect
> > >>> > > > > > > > > established connections?
> > >>> > > > > > > > >
> > >>> > > > > > > > > I'm not familiar with using Storm's DRPC feature, but
> > I'm
> > >>> > sure
> > >>> > > > you
> > >>> > > > > > need
> > >>> > > > > > > > to
> > >>> > > > > > > > > write code that interacts with the DRPC server, and
> > >>> you've
> > >>> > made
> > >>> > > > no
> > >>> > > > > > > > mention
> > >>> > > > > > > > > of doing so in your email. I'd start here:
> > >>> > > > > > > > >
> > >>> > > > > > > > > -
> > >>> > > https://storm.apache.org/documentation/Distributed-RPC.html
> > >>> > > > > > > > >
> > >>> > > > > > > > > - Erik
> > >>> > > > > > > > >
> > >>> > > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > >>> > > > > > > > prog.researcher@gmail.com <javascript:;>
> <javascript:;>>
> > >>> > > > > > > > > wrote:
> > >>> > > > > > > > >
> > >>> > > > > > > > > > I set in the code
> > >>> > > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > >>> > > > > > > > > > conf.put(Config.DRPC_PORT, 3772);
> > >>> > > > > > > > > > but when i submit topolgoy i found at the end of
> the
> > >>> file
> > >>> > > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC
> > servers...
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > i checked port 3772 by
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > sudo netstat -ap | grep 3772
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > i got
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > tcp 0 0 *:3772 *:* LISTEN
> > >>> 10864/java
> > >>> > > > > > > > > > unix 3 [ ] STREAM CONNECTED
> > 13772
> > >>> > > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > >>> > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > why it's not established ?
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > can i find help ?
> > >>> > > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
thanks but Agian i imported this project from github that it's supposed
work well , really i'm on this problem for months without fixing it !! i
sent coder message but he didn't reply with this problem . don't know who
can i ask but thanks for your time
On Thu, Jan 28, 2016 at 1:12 AM, Erik Weathers <
eweathers@groupon.com.invalid> wrote:
> hey Sam,
>
> Again, I've never used the DRPC feature. But it naively looks like you
> aren't using it correctly.
> Note that the working case for you is with a LocalCluster & LocalDRPC, and
> you are explicitly invoking the execute():
> String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
>
> In the remote case you aren't doing anything except submitting the
> topology. I assume you need to invoke drpc.execute *somewhere*... Here's
> what I think is the same question as you are posing:
>
> - http://stackoverflow.com/a/26440260/318428
>
> - Erik
>
> On Wed, Jan 27, 2016 at 3:04 PM, researcher cs <pr...@gmail.com>
> wrote:
>
> > also got zeros after open storm ui except executed column
> >
> >
> >
> >
> > On Thu, Jan 28, 2016 at 12:43 AM, researcher cs <
> prog.researcher@gmail.com
> > > wrote:
> >
> >> i atteached what i got when i submitted topology and this happened also
> >> when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> >> storm.starter.BasicDRPCTopology basic-drpc
> >>
> >> this storm.yaml
> >> storm.zookeeper.servers:
> >> - "localhost"
> >> # - "server2"
> >> nimbus.host: "localhost"
> >>
> >> storm.local.dir: "/var/storm"
> >> supervisor.slots.ports:
> >> - 7660
> >> - 7659
> >> - 7658
> >> - 7657
> >> supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
> >> nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
> >> worker.childopts: "-Djava.net.preferIPv4Stack=true"
> >> # topology.message.timeout.secs: 30
> >> # topology.workers: 1
> >> topology.stats.sample.rate: 1.0
> >> topology.acker.executors: 1
> >> # topology.executor.receive.buffer.size: 16384
> >> # topology.executor.send.buffer.size: 16384
> >> # topology.transfer.buffer.size: 32
> >> # topology.receiver.buffer.size: 8
> >> #
> >> # ##### These may optionally be filled in:
> >> #
> >> ## List of custom serializations
> >> # topology.kryo.register:
> >> # - org.mycompany.MyType
> >> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer
> >> #
> >> ## List of custom kryo decorators
> >> # topology.kryo.decorators:
> >> # - org.mycompany.MyDecorator
> >> #
> >> ## Locations of the drpc servers
> >> # drpc.servers:
> >> # - "localhost"
> >> # - "server2"
> >>
> >> ## Metrics Consumers
> >> # topology.metrics.consumer.register:
> >> # - class: "backtype.storm.metrics.LoggingMetricsConsumer"
> >> # parallelism.hint: 1
> >> # - class: "org.mycompany.MyMetricsConsumer"
> >> # parallelism.hint: 1
> >> # argument:
> >> # - endpoint: "metrics-collector.mycompany.org"
> >> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
> >> storm.messaging.netty.server_worker_threads: 1
> >> storm.messaging.netty.client_worker_threads: 1
> >> storm.messaging.netty.buffer_size: 5242880
> >> storm.messaging.netty.max_retries: 100
> >> storm.messaging.netty.max_wait_ms: 1000
> >> storm.messaging.netty.min_wait_ms: 100
> >>
> >> this etc/hosts
> >> 127.0.0.1 localhost
> >> 127.0.1.1 user-Lenovo-G50-70
> >> ##The following lines are desirable for IPv6 capable hosts##
> >> ::1 ip6-localhost ip6-loopback
> >> fe00::0 ip6-localnet
> >> ff00::0 ip6-mcastprefix
> >> ff02::1 ip6-allnodes
> >> ff02::2 ip6-allrouters
> >>
> >> i set drpc in code
> >> private static Config createTopologyConfiguration(Properties prop,
> >> boolean localMode) {
> >> Config conf = new Config();
> >> List<String> dprcServers = new ArrayList<String>();
> >> dprcServers.add("localhost");
> >>
> >> conf.put(Config.DRPC_SERVERS, dprcServers);
> >> conf.put(Config.DRPC_PORT, 3772);
> >> if (!localMode)
> >> conf.put(Config.STORM_CLUSTER_MODE, new
> >> String("distributed"));
> >>
> >> }
> >> public static void main(String[] args) throws Exception {
> >> if (args == null || args.length == 0) {
> >> Config conf = createTopologyConfiguration(prop, true);
> >> LocalDRPC drpc = new LocalDRPC();
> >> LocalCluster cluster = new LocalCluster();
> >>
> >> cluster.submitTopology(TOPOLOGY_NAME, conf,
> >> buildTopology(drpc));
> >>
> >> Thread.sleep(2000); // give it some time to setup
> >>
> >> BufferedReader br = new BufferedReader(new
> >> FileReader(queryFile));
> >> String tweetJson;
> >> fos.write("Start: ".getBytes());
> >>
> >> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
> >> byte[] newLine = "\n".getBytes();
> >> int times = 0;
> >> // emit tweets into topology
> >> while ((tweetJson = br.readLine()) != null) {
> >>
> >> String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
> >>
> >> Status s = null;
> >> try {
> >> s = DataObjectFactory.createStatus(tweetJson);
> >> result = s.getId() + "\t" + s.getText() + "\t" +
> >> result;
> >> } catch (TwitterException e) {
> >> LOG.error(e.toString());
> >> }
> >>
> >> fos.write(result.getBytes());
> >> fos.write(newLine);
> >>
> >> // times++;
> >> // if (times == 1000)
> >> // break;
> >> }
> >> fos.write(newLine);
> >> fos.write("Finish: ".getBytes());
> >>
> >> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
> >>
> >> fos.flush();
> >> fos.close();
> >> br.close();
> >> drpc.shutdown();
> >> cluster.shutdown();
> >> } else {
> >> // distributed mode
> >> Config conf = createTopologyConfiguration(prop,true);
> >> LocalDRPC drpc = null;
> >> StormSubmitter.submitTopology(args[0], conf,
> >> buildTopology(drpc));
> >>
> >> }
> >>
> >> }
> >>
> >> On Wed, Jan 27, 2016 at 11:14 PM, Erik Weathers <
> >> eweathers@groupon.com.invalid> wrote:
> >>
> >>> Please put more effort into describing the issue. "It doesn't work" is
> >>> unfortunately not enough info for anyone to provide help.
> >>> e.g., post links to some code you are trying to run, and the configs of
> >>> the
> >>> storm components that you are running.
> >>>
> >>> - Erik
> >>>
> >>> On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sa...@gmail.com>
> wrote:
> >>>
> >>> > I wrote the actual problem in My first message drpc server not
> working
> >>> I
> >>> > hot zeros in the columns of storm ui like emitted and transferred ,
> >>> result
> >>> > file is empty
> >>> >
> >>> > On Wednesday, January 27, 2016, Erik Weathers
> >>> > <ew...@groupon.com.invalid>
> >>> > wrote:
> >>> >
> >>> > > Your mail client is wrapping the log lines prematurely, I have a
> >>> really
> >>> > > really hard time reading wrapped lines, I'd look into fixing that
> if
> >>> I
> >>> > were
> >>> > > you. Here they are unwrapped:
> >>> > >
> >>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> >>> connection,
> >>> > > connectString=localhost:2181 sessionTimeout=20000
> >>> > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> >>> connection to
> >>> > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
> >>> using
> >>> > > SASL (unknown error)
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> >>> established
> >>> > > to localhost/127.0.1.1:2181, initiating session
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> >>> > complete
> >>> > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> >>> > > negotiated timeout = 20000
> >>> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> >>> > > :connected:none
> >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session:
> 0x152804f3a3a0002
> >>> > > closed
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> >>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> >>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> >>> connection,
> >>> > > connectString=localhost:2181/storm sessionTimeout=20000
> >>> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> >>> connection to
> >>> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> >>> > authenticate
> >>> > > using SASL (unknown error)
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> >>> established
> >>> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> >>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> >>> > complete
> >>> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> >>> > > 0x152804f3a3a0003,
> >>> > > negotiated timeout = 20000
> >>> > >
> >>> > > None of those indicate a problem, they look pretty standard to me.
> >>> > >
> >>> > > Please spend a bit more time zeroing in on what the actual problem
> >>> is so
> >>> > > that the members of the list(s) can provide help.
> >>> > >
> >>> > > - Erik
> >>> > >
> >>> > > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
> >>> > prog.researcher@gmail.com
> >>> > > <javascript:;>>
> >>> > > wrote:
> >>> > >
> >>> > > > yes i tried in local and worked well
> >>> > > > and about /etc/hosts . i'm feeling that this file has a mistake
> , i
> >>> > made
> >>> > > > alot of changes in this file and didn't remember what was default
> >>> > > >
> >>> > > > In nimbus log file when it connected zookeeper i got
> >>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO]
> Starting
> >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> >>> > connection,
> >>> > > > connectString=localhost:2181 sessionTimeout=20000
> >>> > > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> >>> connection
> >>> > to
> >>> > > > server localhost/127.0.1.1:2181. Will not attempt to
> authenticate
> >>> > using
> >>> > > > SASL (unknown error)
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> >>> > established
> >>> > > > to localhost/127.0.1.1:2181, initiating session
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> >>> > > complete
> >>> > > > on server localhost/127.0.1.1:2181, sessionid =
> 0x152804f3a3a0002,
> >>> > > > negotiated timeout = 20000
> >>> > > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> >>> > > > :connected:none
> >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session:
> >>> 0x152804f3a3a0002
> >>> > > > closed
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> >>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO]
> Starting
> >>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> >>> > connection,
> >>> > > > connectString=localhost:2181/storm sessionTimeout=20000
> >>> > > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
> >>> connection
> >>> > to
> >>> > > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> >>> > > authenticate
> >>> > > > using SASL (unknown error)
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> >>> > established
> >>> > > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> >>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> >>> > > complete
> >>> > > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> >>> > > > 0x152804f3a3a0003,
> >>> > > > negotiated timeout = 20000
> >>> > > >
> >>> > > > and i set for drpc.server : localhost
> >>> > > > storm.zookeeper.server : localhost
> >>> > > > nimbus.host : localhost
> >>> > > >
> >>> > > > in my /etc/hosts
> >>> > > > 127.0.0.1 user-Lenovo-G50-70 localhost
> >>> > > > 127.0.1.1 localhost
> >>> > > >
> >>> > > > is that right ?
> >>> > > >
> >>> > > >
> >>> > > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
> >>> > > > eweathers@groupon.com.invalid> wrote:
> >>> > > >
> >>> > > > > You said: "except the statement of drpc server trying to
> connect"
> >>> > > > >
> >>> > > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
> >>> > > Distributed
> >>> > > > > RPC servers..." implies?
> >>> > > > > That is just saying that the server is being started. It's a
> >>> > server,
> >>> > > > not
> >>> > > > > a client, so it's basic operation is *not* to connect to some
> >>> other
> >>> > > > thing.
> >>> > > > > It's up and waiting for you to tell it to do stuff.
> >>> > > > >
> >>> > > > > Have you gotten the Local-mode version of DRPC working?
> >>> > > > >
> >>> > > > > -
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
> >>> > > > >
> >>> > > > > Not sure what you're asking with regards to DNS and /etc/hosts,
> >>> those
> >>> > > > seem
> >>> > > > > unrelated to your basic issue.
> >>> > > > >
> >>> > > > > - Erik
> >>> > > > >
> >>> > > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
> >>> > > > prog.researcher@gmail.com <javascript:;>>
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > There is no error except the statement of drpc server trying
> to
> >>> > > connect
> >>> > > > > . I
> >>> > > > > > guess I have problem with Dns . if you have any idea about
> this
> >>> > > pleaes
> >>> > > > > help
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > I want to submit topology with single machine
> >>> > > > > > Only on my laptop without any other devices as a first step
> so
> >>> > > > > > What this file /etc/hosts should contains ?
> >>> > > > > > As I set in drpc.server : localhost
> >>> > > > > > Storm.zookeeper. server: localhost
> >>> > > > > > Nimbus.host:localhost
> >>> > > > > >
> >>> > > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP
> >>> address ?
> >>> > > > > >
> >>> > > > > > What should I use and what should I hash it to not using it ?
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > On Wednesday, January 27, 2016, Erik Weathers
> >>> > > > > > <ew...@groupon.com.invalid>
> >>> > > > > > wrote:
> >>> > > > > >
> >>> > > > > > > What does the client code that is supposed to make the DRPC
> >>> > > > connection
> >>> > > > > > > telling you? i.e., you should see some exception or log
> >>> about
> >>> > not
> >>> > > > > > > establishing the connection, right?
> >>> > > > > > >
> >>> > > > > > > Alternatively, perhaps the connections aren't persistent
> and
> >>> > > there's
> >>> > > > no
> >>> > > > > > > actual problem?
> >>> > > > > > >
> >>> > > > > > > - Erik
> >>> > > > > > >
> >>> > > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> >>> > > > > > prog.researcher@gmail.com <javascript:;>
> >>> > > > > > > <javascript:;>>
> >>> > > > > > > wrote:
> >>> > > > > > >
> >>> > > > > > > > thanks for replying , i read the documentation before ,
> i
> >>> > > imported
> >>> > > > > > > project
> >>> > > > > > > > supposed to work well but not working with me
> >>> > > > > > > > i checked port by lsof -i gave me all ports i connected
> it
> >>> for
> >>> > > > storm
> >>> > > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP
> >>> *:52022
> >>> > > > > > (LISTEN)
> >>> > > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP
> >>> *:2181
> >>> > > > > (LISTEN)
> >>> > > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
> >>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> >>> > > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
> >>> > > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526
> >>> (ESTABLISHED)
> >>> > > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
> >>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> >>> > > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
> >>> > > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> >>> > > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP
> >>> *:6627
> >>> > > > > (LISTEN)
> >>> > > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
> >>> > > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181
> >>> (ESTABLISHED)
> >>> > > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP
> >>> *:3772
> >>> > > > > (LISTEN)
> >>> > > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP
> >>> *:3773
> >>> > > > > (LISTEN)
> >>> > > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
> >>> > > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825
> >>> (ESTABLISHED)
> >>> > > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
> >>> > > *:http-alt
> >>> > > > > > > (LISTEN)
> >>> > > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
> >>> > > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773
> >>> (ESTABLISHED)
> >>> > > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
> >>> > > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> >>> > > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP
> >>> *:7660
> >>> > > > > (LISTEN)
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > here you can see that 3772 not established
> >>> > > > > > > >
> >>> > > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> >>> > > > > > > > eweathers@groupon.com.invalid> wrote:
> >>> > > > > > > >
> >>> > > > > > > > > hey,
> >>> > > > > > > > >
> >>> > > > > > > > > The DRPC server is up and listening on port 3772. Why
> >>> do
> >>> > you
> >>> > > > > expect
> >>> > > > > > > > > established connections?
> >>> > > > > > > > >
> >>> > > > > > > > > I'm not familiar with using Storm's DRPC feature, but
> I'm
> >>> > sure
> >>> > > > you
> >>> > > > > > need
> >>> > > > > > > > to
> >>> > > > > > > > > write code that interacts with the DRPC server, and
> >>> you've
> >>> > made
> >>> > > > no
> >>> > > > > > > > mention
> >>> > > > > > > > > of doing so in your email. I'd start here:
> >>> > > > > > > > >
> >>> > > > > > > > > -
> >>> > > https://storm.apache.org/documentation/Distributed-RPC.html
> >>> > > > > > > > >
> >>> > > > > > > > > - Erik
> >>> > > > > > > > >
> >>> > > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> >>> > > > > > > > prog.researcher@gmail.com <javascript:;> <javascript:;>>
> >>> > > > > > > > > wrote:
> >>> > > > > > > > >
> >>> > > > > > > > > > I set in the code
> >>> > > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> >>> > > > > > > > > > conf.put(Config.DRPC_PORT, 3772);
> >>> > > > > > > > > > but when i submit topolgoy i found at the end of the
> >>> file
> >>> > > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC
> servers...
> >>> > > > > > > > > >
> >>> > > > > > > > > > i checked port 3772 by
> >>> > > > > > > > > >
> >>> > > > > > > > > > sudo netstat -ap | grep 3772
> >>> > > > > > > > > >
> >>> > > > > > > > > > i got
> >>> > > > > > > > > >
> >>> > > > > > > > > > tcp 0 0 *:3772 *:* LISTEN
> >>> 10864/java
> >>> > > > > > > > > > unix 3 [ ] STREAM CONNECTED
> 13772
> >>> > > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> >>> > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > > > > why it's not established ?
> >>> > > > > > > > > >
> >>> > > > > > > > > > can i find help ?
> >>> > > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
hey Sam,
Again, I've never used the DRPC feature. But it naively looks like you
aren't using it correctly.
Note that the working case for you is with a LocalCluster & LocalDRPC, and
you are explicitly invoking the execute():
String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
In the remote case you aren't doing anything except submitting the
topology. I assume you need to invoke drpc.execute *somewhere*... Here's
what I think is the same question as you are posing:
- http://stackoverflow.com/a/26440260/318428
- Erik
On Wed, Jan 27, 2016 at 3:04 PM, researcher cs <pr...@gmail.com>
wrote:
> also got zeros after open storm ui except executed column
>
>
>
>
> On Thu, Jan 28, 2016 at 12:43 AM, researcher cs <prog.researcher@gmail.com
> > wrote:
>
>> i atteached what i got when i submitted topology and this happened also
>> when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
>> storm.starter.BasicDRPCTopology basic-drpc
>>
>> this storm.yaml
>> storm.zookeeper.servers:
>> - "localhost"
>> # - "server2"
>> nimbus.host: "localhost"
>>
>> storm.local.dir: "/var/storm"
>> supervisor.slots.ports:
>> - 7660
>> - 7659
>> - 7658
>> - 7657
>> supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
>> nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
>> worker.childopts: "-Djava.net.preferIPv4Stack=true"
>> # topology.message.timeout.secs: 30
>> # topology.workers: 1
>> topology.stats.sample.rate: 1.0
>> topology.acker.executors: 1
>> # topology.executor.receive.buffer.size: 16384
>> # topology.executor.send.buffer.size: 16384
>> # topology.transfer.buffer.size: 32
>> # topology.receiver.buffer.size: 8
>> #
>> # ##### These may optionally be filled in:
>> #
>> ## List of custom serializations
>> # topology.kryo.register:
>> # - org.mycompany.MyType
>> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer
>> #
>> ## List of custom kryo decorators
>> # topology.kryo.decorators:
>> # - org.mycompany.MyDecorator
>> #
>> ## Locations of the drpc servers
>> # drpc.servers:
>> # - "localhost"
>> # - "server2"
>>
>> ## Metrics Consumers
>> # topology.metrics.consumer.register:
>> # - class: "backtype.storm.metrics.LoggingMetricsConsumer"
>> # parallelism.hint: 1
>> # - class: "org.mycompany.MyMetricsConsumer"
>> # parallelism.hint: 1
>> # argument:
>> # - endpoint: "metrics-collector.mycompany.org"
>> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
>> storm.messaging.netty.server_worker_threads: 1
>> storm.messaging.netty.client_worker_threads: 1
>> storm.messaging.netty.buffer_size: 5242880
>> storm.messaging.netty.max_retries: 100
>> storm.messaging.netty.max_wait_ms: 1000
>> storm.messaging.netty.min_wait_ms: 100
>>
>> this etc/hosts
>> 127.0.0.1 localhost
>> 127.0.1.1 user-Lenovo-G50-70
>> ##The following lines are desirable for IPv6 capable hosts##
>> ::1 ip6-localhost ip6-loopback
>> fe00::0 ip6-localnet
>> ff00::0 ip6-mcastprefix
>> ff02::1 ip6-allnodes
>> ff02::2 ip6-allrouters
>>
>> i set drpc in code
>> private static Config createTopologyConfiguration(Properties prop,
>> boolean localMode) {
>> Config conf = new Config();
>> List<String> dprcServers = new ArrayList<String>();
>> dprcServers.add("localhost");
>>
>> conf.put(Config.DRPC_SERVERS, dprcServers);
>> conf.put(Config.DRPC_PORT, 3772);
>> if (!localMode)
>> conf.put(Config.STORM_CLUSTER_MODE, new
>> String("distributed"));
>>
>> }
>> public static void main(String[] args) throws Exception {
>> if (args == null || args.length == 0) {
>> Config conf = createTopologyConfiguration(prop, true);
>> LocalDRPC drpc = new LocalDRPC();
>> LocalCluster cluster = new LocalCluster();
>>
>> cluster.submitTopology(TOPOLOGY_NAME, conf,
>> buildTopology(drpc));
>>
>> Thread.sleep(2000); // give it some time to setup
>>
>> BufferedReader br = new BufferedReader(new
>> FileReader(queryFile));
>> String tweetJson;
>> fos.write("Start: ".getBytes());
>>
>> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
>> byte[] newLine = "\n".getBytes();
>> int times = 0;
>> // emit tweets into topology
>> while ((tweetJson = br.readLine()) != null) {
>>
>> String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
>>
>> Status s = null;
>> try {
>> s = DataObjectFactory.createStatus(tweetJson);
>> result = s.getId() + "\t" + s.getText() + "\t" +
>> result;
>> } catch (TwitterException e) {
>> LOG.error(e.toString());
>> }
>>
>> fos.write(result.getBytes());
>> fos.write(newLine);
>>
>> // times++;
>> // if (times == 1000)
>> // break;
>> }
>> fos.write(newLine);
>> fos.write("Finish: ".getBytes());
>>
>> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
>>
>> fos.flush();
>> fos.close();
>> br.close();
>> drpc.shutdown();
>> cluster.shutdown();
>> } else {
>> // distributed mode
>> Config conf = createTopologyConfiguration(prop,true);
>> LocalDRPC drpc = null;
>> StormSubmitter.submitTopology(args[0], conf,
>> buildTopology(drpc));
>>
>> }
>>
>> }
>>
>> On Wed, Jan 27, 2016 at 11:14 PM, Erik Weathers <
>> eweathers@groupon.com.invalid> wrote:
>>
>>> Please put more effort into describing the issue. "It doesn't work" is
>>> unfortunately not enough info for anyone to provide help.
>>> e.g., post links to some code you are trying to run, and the configs of
>>> the
>>> storm components that you are running.
>>>
>>> - Erik
>>>
>>> On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sa...@gmail.com> wrote:
>>>
>>> > I wrote the actual problem in My first message drpc server not working
>>> I
>>> > hot zeros in the columns of storm ui like emitted and transferred ,
>>> result
>>> > file is empty
>>> >
>>> > On Wednesday, January 27, 2016, Erik Weathers
>>> > <ew...@groupon.com.invalid>
>>> > wrote:
>>> >
>>> > > Your mail client is wrapping the log lines prematurely, I have a
>>> really
>>> > > really hard time reading wrapped lines, I'd look into fixing that if
>>> I
>>> > were
>>> > > you. Here they are unwrapped:
>>> > >
>>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>>> connection,
>>> > > connectString=localhost:2181 sessionTimeout=20000
>>> > > watcher=com.netflix.curator.ConnectionState@2fa423d2
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
>>> connection to
>>> > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
>>> using
>>> > > SASL (unknown error)
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>>> established
>>> > > to localhost/127.0.1.1:2181, initiating session
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>>> > complete
>>> > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
>>> > > negotiated timeout = 20000
>>> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
>>> > > :connected:none
>>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
>>> > > closed
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
>>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>>> connection,
>>> > > connectString=localhost:2181/storm sessionTimeout=20000
>>> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
>>> connection to
>>> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
>>> > authenticate
>>> > > using SASL (unknown error)
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>>> established
>>> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
>>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>>> > complete
>>> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
>>> > > 0x152804f3a3a0003,
>>> > > negotiated timeout = 20000
>>> > >
>>> > > None of those indicate a problem, they look pretty standard to me.
>>> > >
>>> > > Please spend a bit more time zeroing in on what the actual problem
>>> is so
>>> > > that the members of the list(s) can provide help.
>>> > >
>>> > > - Erik
>>> > >
>>> > > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
>>> > prog.researcher@gmail.com
>>> > > <javascript:;>>
>>> > > wrote:
>>> > >
>>> > > > yes i tried in local and worked well
>>> > > > and about /etc/hosts . i'm feeling that this file has a mistake , i
>>> > made
>>> > > > alot of changes in this file and didn't remember what was default
>>> > > >
>>> > > > In nimbus log file when it connected zookeeper i got
>>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>>> > connection,
>>> > > > connectString=localhost:2181 sessionTimeout=20000
>>> > > > watcher=com.netflix.curator.ConnectionState@2fa423d2
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
>>> connection
>>> > to
>>> > > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
>>> > using
>>> > > > SASL (unknown error)
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>>> > established
>>> > > > to localhost/127.0.1.1:2181, initiating session
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>>> > > complete
>>> > > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
>>> > > > negotiated timeout = 20000
>>> > > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
>>> > > > :connected:none
>>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session:
>>> 0x152804f3a3a0002
>>> > > > closed
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
>>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>>> > connection,
>>> > > > connectString=localhost:2181/storm sessionTimeout=20000
>>> > > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
>>> connection
>>> > to
>>> > > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
>>> > > authenticate
>>> > > > using SASL (unknown error)
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>>> > established
>>> > > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
>>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>>> > > complete
>>> > > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
>>> > > > 0x152804f3a3a0003,
>>> > > > negotiated timeout = 20000
>>> > > >
>>> > > > and i set for drpc.server : localhost
>>> > > > storm.zookeeper.server : localhost
>>> > > > nimbus.host : localhost
>>> > > >
>>> > > > in my /etc/hosts
>>> > > > 127.0.0.1 user-Lenovo-G50-70 localhost
>>> > > > 127.0.1.1 localhost
>>> > > >
>>> > > > is that right ?
>>> > > >
>>> > > >
>>> > > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
>>> > > > eweathers@groupon.com.invalid> wrote:
>>> > > >
>>> > > > > You said: "except the statement of drpc server trying to connect"
>>> > > > >
>>> > > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
>>> > > Distributed
>>> > > > > RPC servers..." implies?
>>> > > > > That is just saying that the server is being started. It's a
>>> > server,
>>> > > > not
>>> > > > > a client, so it's basic operation is *not* to connect to some
>>> other
>>> > > > thing.
>>> > > > > It's up and waiting for you to tell it to do stuff.
>>> > > > >
>>> > > > > Have you gotten the Local-mode version of DRPC working?
>>> > > > >
>>> > > > > -
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
>>> > > > >
>>> > > > > Not sure what you're asking with regards to DNS and /etc/hosts,
>>> those
>>> > > > seem
>>> > > > > unrelated to your basic issue.
>>> > > > >
>>> > > > > - Erik
>>> > > > >
>>> > > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
>>> > > > prog.researcher@gmail.com <javascript:;>>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > There is no error except the statement of drpc server trying to
>>> > > connect
>>> > > > > . I
>>> > > > > > guess I have problem with Dns . if you have any idea about this
>>> > > pleaes
>>> > > > > help
>>> > > > > >
>>> > > > > >
>>> > > > > > I want to submit topology with single machine
>>> > > > > > Only on my laptop without any other devices as a first step so
>>> > > > > > What this file /etc/hosts should contains ?
>>> > > > > > As I set in drpc.server : localhost
>>> > > > > > Storm.zookeeper. server: localhost
>>> > > > > > Nimbus.host:localhost
>>> > > > > >
>>> > > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP
>>> address ?
>>> > > > > >
>>> > > > > > What should I use and what should I hash it to not using it ?
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > On Wednesday, January 27, 2016, Erik Weathers
>>> > > > > > <ew...@groupon.com.invalid>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > > > What does the client code that is supposed to make the DRPC
>>> > > > connection
>>> > > > > > > telling you? i.e., you should see some exception or log
>>> about
>>> > not
>>> > > > > > > establishing the connection, right?
>>> > > > > > >
>>> > > > > > > Alternatively, perhaps the connections aren't persistent and
>>> > > there's
>>> > > > no
>>> > > > > > > actual problem?
>>> > > > > > >
>>> > > > > > > - Erik
>>> > > > > > >
>>> > > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
>>> > > > > > prog.researcher@gmail.com <javascript:;>
>>> > > > > > > <javascript:;>>
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > thanks for replying , i read the documentation before , i
>>> > > imported
>>> > > > > > > project
>>> > > > > > > > supposed to work well but not working with me
>>> > > > > > > > i checked port by lsof -i gave me all ports i connected it
>>> for
>>> > > > storm
>>> > > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP
>>> *:52022
>>> > > > > > (LISTEN)
>>> > > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP
>>> *:2181
>>> > > > > (LISTEN)
>>> > > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
>>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
>>> > > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
>>> > > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526
>>> (ESTABLISHED)
>>> > > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
>>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
>>> > > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
>>> > > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
>>> > > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP
>>> *:6627
>>> > > > > (LISTEN)
>>> > > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
>>> > > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181
>>> (ESTABLISHED)
>>> > > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP
>>> *:3772
>>> > > > > (LISTEN)
>>> > > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP
>>> *:3773
>>> > > > > (LISTEN)
>>> > > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
>>> > > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825
>>> (ESTABLISHED)
>>> > > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
>>> > > *:http-alt
>>> > > > > > > (LISTEN)
>>> > > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
>>> > > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773
>>> (ESTABLISHED)
>>> > > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
>>> > > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
>>> > > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP
>>> *:7660
>>> > > > > (LISTEN)
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > here you can see that 3772 not established
>>> > > > > > > >
>>> > > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
>>> > > > > > > > eweathers@groupon.com.invalid> wrote:
>>> > > > > > > >
>>> > > > > > > > > hey,
>>> > > > > > > > >
>>> > > > > > > > > The DRPC server is up and listening on port 3772. Why
>>> do
>>> > you
>>> > > > > expect
>>> > > > > > > > > established connections?
>>> > > > > > > > >
>>> > > > > > > > > I'm not familiar with using Storm's DRPC feature, but I'm
>>> > sure
>>> > > > you
>>> > > > > > need
>>> > > > > > > > to
>>> > > > > > > > > write code that interacts with the DRPC server, and
>>> you've
>>> > made
>>> > > > no
>>> > > > > > > > mention
>>> > > > > > > > > of doing so in your email. I'd start here:
>>> > > > > > > > >
>>> > > > > > > > > -
>>> > > https://storm.apache.org/documentation/Distributed-RPC.html
>>> > > > > > > > >
>>> > > > > > > > > - Erik
>>> > > > > > > > >
>>> > > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
>>> > > > > > > > prog.researcher@gmail.com <javascript:;> <javascript:;>>
>>> > > > > > > > > wrote:
>>> > > > > > > > >
>>> > > > > > > > > > I set in the code
>>> > > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
>>> > > > > > > > > > conf.put(Config.DRPC_PORT, 3772);
>>> > > > > > > > > > but when i submit topolgoy i found at the end of the
>>> file
>>> > > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
>>> > > > > > > > > >
>>> > > > > > > > > > i checked port 3772 by
>>> > > > > > > > > >
>>> > > > > > > > > > sudo netstat -ap | grep 3772
>>> > > > > > > > > >
>>> > > > > > > > > > i got
>>> > > > > > > > > >
>>> > > > > > > > > > tcp 0 0 *:3772 *:* LISTEN
>>> 10864/java
>>> > > > > > > > > > unix 3 [ ] STREAM CONNECTED 13772
>>> > > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > why it's not established ?
>>> > > > > > > > > >
>>> > > > > > > > > > can i find help ?
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
also got zeros after open storm ui except executed column
On Thu, Jan 28, 2016 at 12:43 AM, researcher cs <pr...@gmail.com>
wrote:
> i atteached what i got when i submitted topology and this happened also
> when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> storm.starter.BasicDRPCTopology basic-drpc
>
> this storm.yaml
> storm.zookeeper.servers:
> - "localhost"
> # - "server2"
> nimbus.host: "localhost"
>
> storm.local.dir: "/var/storm"
> supervisor.slots.ports:
> - 7660
> - 7659
> - 7658
> - 7657
> supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
> nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
> worker.childopts: "-Djava.net.preferIPv4Stack=true"
> # topology.message.timeout.secs: 30
> # topology.workers: 1
> topology.stats.sample.rate: 1.0
> topology.acker.executors: 1
> # topology.executor.receive.buffer.size: 16384
> # topology.executor.send.buffer.size: 16384
> # topology.transfer.buffer.size: 32
> # topology.receiver.buffer.size: 8
> #
> # ##### These may optionally be filled in:
> #
> ## List of custom serializations
> # topology.kryo.register:
> # - org.mycompany.MyType
> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer
> #
> ## List of custom kryo decorators
> # topology.kryo.decorators:
> # - org.mycompany.MyDecorator
> #
> ## Locations of the drpc servers
> # drpc.servers:
> # - "localhost"
> # - "server2"
>
> ## Metrics Consumers
> # topology.metrics.consumer.register:
> # - class: "backtype.storm.metrics.LoggingMetricsConsumer"
> # parallelism.hint: 1
> # - class: "org.mycompany.MyMetricsConsumer"
> # parallelism.hint: 1
> # argument:
> # - endpoint: "metrics-collector.mycompany.org"
> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
> storm.messaging.netty.server_worker_threads: 1
> storm.messaging.netty.client_worker_threads: 1
> storm.messaging.netty.buffer_size: 5242880
> storm.messaging.netty.max_retries: 100
> storm.messaging.netty.max_wait_ms: 1000
> storm.messaging.netty.min_wait_ms: 100
>
> this etc/hosts
> 127.0.0.1 localhost
> 127.0.1.1 user-Lenovo-G50-70
> ##The following lines are desirable for IPv6 capable hosts##
> ::1 ip6-localhost ip6-loopback
> fe00::0 ip6-localnet
> ff00::0 ip6-mcastprefix
> ff02::1 ip6-allnodes
> ff02::2 ip6-allrouters
>
> i set drpc in code
> private static Config createTopologyConfiguration(Properties prop,
> boolean localMode) {
> Config conf = new Config();
> List<String> dprcServers = new ArrayList<String>();
> dprcServers.add("localhost");
>
> conf.put(Config.DRPC_SERVERS, dprcServers);
> conf.put(Config.DRPC_PORT, 3772);
> if (!localMode)
> conf.put(Config.STORM_CLUSTER_MODE, new String("distributed"));
>
> }
> public static void main(String[] args) throws Exception {
> if (args == null || args.length == 0) {
> Config conf = createTopologyConfiguration(prop, true);
> LocalDRPC drpc = new LocalDRPC();
> LocalCluster cluster = new LocalCluster();
>
> cluster.submitTopology(TOPOLOGY_NAME, conf,
> buildTopology(drpc));
>
> Thread.sleep(2000); // give it some time to setup
>
> BufferedReader br = new BufferedReader(new
> FileReader(queryFile));
> String tweetJson;
> fos.write("Start: ".getBytes());
>
> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
> byte[] newLine = "\n".getBytes();
> int times = 0;
> // emit tweets into topology
> while ((tweetJson = br.readLine()) != null) {
>
> String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
>
> Status s = null;
> try {
> s = DataObjectFactory.createStatus(tweetJson);
> result = s.getId() + "\t" + s.getText() + "\t" +
> result;
> } catch (TwitterException e) {
> LOG.error(e.toString());
> }
>
> fos.write(result.getBytes());
> fos.write(newLine);
>
> // times++;
> // if (times == 1000)
> // break;
> }
> fos.write(newLine);
> fos.write("Finish: ".getBytes());
>
> fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
>
> fos.flush();
> fos.close();
> br.close();
> drpc.shutdown();
> cluster.shutdown();
> } else {
> // distributed mode
> Config conf = createTopologyConfiguration(prop,true);
> LocalDRPC drpc = null;
> StormSubmitter.submitTopology(args[0], conf,
> buildTopology(drpc));
>
> }
>
> }
>
> On Wed, Jan 27, 2016 at 11:14 PM, Erik Weathers <
> eweathers@groupon.com.invalid> wrote:
>
>> Please put more effort into describing the issue. "It doesn't work" is
>> unfortunately not enough info for anyone to provide help.
>> e.g., post links to some code you are trying to run, and the configs of
>> the
>> storm components that you are running.
>>
>> - Erik
>>
>> On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sa...@gmail.com> wrote:
>>
>> > I wrote the actual problem in My first message drpc server not working I
>> > hot zeros in the columns of storm ui like emitted and transferred ,
>> result
>> > file is empty
>> >
>> > On Wednesday, January 27, 2016, Erik Weathers
>> > <ew...@groupon.com.invalid>
>> > wrote:
>> >
>> > > Your mail client is wrapping the log lines prematurely, I have a
>> really
>> > > really hard time reading wrapped lines, I'd look into fixing that if I
>> > were
>> > > you. Here they are unwrapped:
>> > >
>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>> connection,
>> > > connectString=localhost:2181 sessionTimeout=20000
>> > > watcher=com.netflix.curator.ConnectionState@2fa423d2
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
>> to
>> > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
>> using
>> > > SASL (unknown error)
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>> established
>> > > to localhost/127.0.1.1:2181, initiating session
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>> > complete
>> > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
>> > > negotiated timeout = 20000
>> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
>> > > :connected:none
>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
>> > > closed
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
>> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>> connection,
>> > > connectString=localhost:2181/storm sessionTimeout=20000
>> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
>> to
>> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
>> > authenticate
>> > > using SASL (unknown error)
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>> established
>> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
>> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>> > complete
>> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
>> > > 0x152804f3a3a0003,
>> > > negotiated timeout = 20000
>> > >
>> > > None of those indicate a problem, they look pretty standard to me.
>> > >
>> > > Please spend a bit more time zeroing in on what the actual problem is
>> so
>> > > that the members of the list(s) can provide help.
>> > >
>> > > - Erik
>> > >
>> > > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
>> > prog.researcher@gmail.com
>> > > <javascript:;>>
>> > > wrote:
>> > >
>> > > > yes i tried in local and worked well
>> > > > and about /etc/hosts . i'm feeling that this file has a mistake , i
>> > made
>> > > > alot of changes in this file and didn't remember what was default
>> > > >
>> > > > In nimbus log file when it connected zookeeper i got
>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>> > connection,
>> > > > connectString=localhost:2181 sessionTimeout=20000
>> > > > watcher=com.netflix.curator.ConnectionState@2fa423d2
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
>> connection
>> > to
>> > > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
>> > using
>> > > > SASL (unknown error)
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>> > established
>> > > > to localhost/127.0.1.1:2181, initiating session
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>> > > complete
>> > > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
>> > > > negotiated timeout = 20000
>> > > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
>> > > > :connected:none
>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session:
>> 0x152804f3a3a0002
>> > > > closed
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
>> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
>> > connection,
>> > > > connectString=localhost:2181/storm sessionTimeout=20000
>> > > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket
>> connection
>> > to
>> > > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
>> > > authenticate
>> > > > using SASL (unknown error)
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
>> > established
>> > > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
>> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
>> > > complete
>> > > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
>> > > > 0x152804f3a3a0003,
>> > > > negotiated timeout = 20000
>> > > >
>> > > > and i set for drpc.server : localhost
>> > > > storm.zookeeper.server : localhost
>> > > > nimbus.host : localhost
>> > > >
>> > > > in my /etc/hosts
>> > > > 127.0.0.1 user-Lenovo-G50-70 localhost
>> > > > 127.0.1.1 localhost
>> > > >
>> > > > is that right ?
>> > > >
>> > > >
>> > > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
>> > > > eweathers@groupon.com.invalid> wrote:
>> > > >
>> > > > > You said: "except the statement of drpc server trying to connect"
>> > > > >
>> > > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
>> > > Distributed
>> > > > > RPC servers..." implies?
>> > > > > That is just saying that the server is being started. It's a
>> > server,
>> > > > not
>> > > > > a client, so it's basic operation is *not* to connect to some
>> other
>> > > > thing.
>> > > > > It's up and waiting for you to tell it to do stuff.
>> > > > >
>> > > > > Have you gotten the Local-mode version of DRPC working?
>> > > > >
>> > > > > -
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
>> > > > >
>> > > > > Not sure what you're asking with regards to DNS and /etc/hosts,
>> those
>> > > > seem
>> > > > > unrelated to your basic issue.
>> > > > >
>> > > > > - Erik
>> > > > >
>> > > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
>> > > > prog.researcher@gmail.com <javascript:;>>
>> > > > > wrote:
>> > > > >
>> > > > > > There is no error except the statement of drpc server trying to
>> > > connect
>> > > > > . I
>> > > > > > guess I have problem with Dns . if you have any idea about this
>> > > pleaes
>> > > > > help
>> > > > > >
>> > > > > >
>> > > > > > I want to submit topology with single machine
>> > > > > > Only on my laptop without any other devices as a first step so
>> > > > > > What this file /etc/hosts should contains ?
>> > > > > > As I set in drpc.server : localhost
>> > > > > > Storm.zookeeper. server: localhost
>> > > > > > Nimbus.host:localhost
>> > > > > >
>> > > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address
>> ?
>> > > > > >
>> > > > > > What should I use and what should I hash it to not using it ?
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Wednesday, January 27, 2016, Erik Weathers
>> > > > > > <ew...@groupon.com.invalid>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > What does the client code that is supposed to make the DRPC
>> > > > connection
>> > > > > > > telling you? i.e., you should see some exception or log about
>> > not
>> > > > > > > establishing the connection, right?
>> > > > > > >
>> > > > > > > Alternatively, perhaps the connections aren't persistent and
>> > > there's
>> > > > no
>> > > > > > > actual problem?
>> > > > > > >
>> > > > > > > - Erik
>> > > > > > >
>> > > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
>> > > > > > prog.researcher@gmail.com <javascript:;>
>> > > > > > > <javascript:;>>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > thanks for replying , i read the documentation before , i
>> > > imported
>> > > > > > > project
>> > > > > > > > supposed to work well but not working with me
>> > > > > > > > i checked port by lsof -i gave me all ports i connected it
>> for
>> > > > storm
>> > > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP
>> *:52022
>> > > > > > (LISTEN)
>> > > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP
>> *:2181
>> > > > > (LISTEN)
>> > > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
>> > > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
>> > > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526
>> (ESTABLISHED)
>> > > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
>> > > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
>> > > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
>> > > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
>> > > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP
>> *:6627
>> > > > > (LISTEN)
>> > > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
>> > > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181
>> (ESTABLISHED)
>> > > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP
>> *:3772
>> > > > > (LISTEN)
>> > > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP
>> *:3773
>> > > > > (LISTEN)
>> > > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
>> > > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825
>> (ESTABLISHED)
>> > > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
>> > > *:http-alt
>> > > > > > > (LISTEN)
>> > > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
>> > > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773
>> (ESTABLISHED)
>> > > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
>> > > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
>> > > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP
>> *:7660
>> > > > > (LISTEN)
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > here you can see that 3772 not established
>> > > > > > > >
>> > > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
>> > > > > > > > eweathers@groupon.com.invalid> wrote:
>> > > > > > > >
>> > > > > > > > > hey,
>> > > > > > > > >
>> > > > > > > > > The DRPC server is up and listening on port 3772. Why do
>> > you
>> > > > > expect
>> > > > > > > > > established connections?
>> > > > > > > > >
>> > > > > > > > > I'm not familiar with using Storm's DRPC feature, but I'm
>> > sure
>> > > > you
>> > > > > > need
>> > > > > > > > to
>> > > > > > > > > write code that interacts with the DRPC server, and you've
>> > made
>> > > > no
>> > > > > > > > mention
>> > > > > > > > > of doing so in your email. I'd start here:
>> > > > > > > > >
>> > > > > > > > > -
>> > > https://storm.apache.org/documentation/Distributed-RPC.html
>> > > > > > > > >
>> > > > > > > > > - Erik
>> > > > > > > > >
>> > > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
>> > > > > > > > prog.researcher@gmail.com <javascript:;> <javascript:;>>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > I set in the code
>> > > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
>> > > > > > > > > > conf.put(Config.DRPC_PORT, 3772);
>> > > > > > > > > > but when i submit topolgoy i found at the end of the
>> file
>> > > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
>> > > > > > > > > >
>> > > > > > > > > > i checked port 3772 by
>> > > > > > > > > >
>> > > > > > > > > > sudo netstat -ap | grep 3772
>> > > > > > > > > >
>> > > > > > > > > > i got
>> > > > > > > > > >
>> > > > > > > > > > tcp 0 0 *:3772 *:* LISTEN
>> 10864/java
>> > > > > > > > > > unix 3 [ ] STREAM CONNECTED 13772
>> > > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > why it's not established ?
>> > > > > > > > > >
>> > > > > > > > > > can i find help ?
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
i atteached what i got when i submitted topology and this happened also
when i submitted storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
storm.starter.BasicDRPCTopology basic-drpc
this storm.yaml
storm.zookeeper.servers:
- "localhost"
# - "server2"
nimbus.host: "localhost"
storm.local.dir: "/var/storm"
supervisor.slots.ports:
- 7660
- 7659
- 7658
- 7657
supervisor.childopts: "-Djava.net.preferIPv4Stack=true"
nimbus.childopts: "-Djava.net.preferIPv4Stack=true"
worker.childopts: "-Djava.net.preferIPv4Stack=true"
# topology.message.timeout.secs: 30
# topology.workers: 1
topology.stats.sample.rate: 1.0
topology.acker.executors: 1
# topology.executor.receive.buffer.size: 16384
# topology.executor.send.buffer.size: 16384
# topology.transfer.buffer.size: 32
# topology.receiver.buffer.size: 8
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "localhost"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "backtype.storm.metrics.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100
this etc/hosts
127.0.0.1 localhost
127.0.1.1 user-Lenovo-G50-70
##The following lines are desirable for IPv6 capable hosts##
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
i set drpc in code
private static Config createTopologyConfiguration(Properties prop,
boolean localMode) {
Config conf = new Config();
List<String> dprcServers = new ArrayList<String>();
dprcServers.add("localhost");
conf.put(Config.DRPC_SERVERS, dprcServers);
conf.put(Config.DRPC_PORT, 3772);
if (!localMode)
conf.put(Config.STORM_CLUSTER_MODE, new String("distributed"));
}
public static void main(String[] args) throws Exception {
if (args == null || args.length == 0) {
Config conf = createTopologyConfiguration(prop, true);
LocalDRPC drpc = new LocalDRPC();
LocalCluster cluster = new LocalCluster();
cluster.submitTopology(TOPOLOGY_NAME, conf,
buildTopology(drpc));
Thread.sleep(2000); // give it some time to setup
BufferedReader br = new BufferedReader(new
FileReader(queryFile));
String tweetJson;
fos.write("Start: ".getBytes());
fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
byte[] newLine = "\n".getBytes();
int times = 0;
// emit tweets into topology
while ((tweetJson = br.readLine()) != null) {
String result = drpc.execute(TOPOLOGY_NAME, tweetJson);
Status s = null;
try {
s = DataObjectFactory.createStatus(tweetJson);
result = s.getId() + "\t" + s.getText() + "\t" + result;
} catch (TwitterException e) {
LOG.error(e.toString());
}
fos.write(result.getBytes());
fos.write(newLine);
// times++;
// if (times == 1000)
// break;
}
fos.write(newLine);
fos.write("Finish: ".getBytes());
fos.write(String.valueOf(System.currentTimeMillis()).getBytes());
fos.flush();
fos.close();
br.close();
drpc.shutdown();
cluster.shutdown();
} else {
// distributed mode
Config conf = createTopologyConfiguration(prop,true);
LocalDRPC drpc = null;
StormSubmitter.submitTopology(args[0], conf,
buildTopology(drpc));
}
}
On Wed, Jan 27, 2016 at 11:14 PM, Erik Weathers <
eweathers@groupon.com.invalid> wrote:
> Please put more effort into describing the issue. "It doesn't work" is
> unfortunately not enough info for anyone to provide help.
> e.g., post links to some code you are trying to run, and the configs of the
> storm components that you are running.
>
> - Erik
>
> On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sa...@gmail.com> wrote:
>
> > I wrote the actual problem in My first message drpc server not working I
> > hot zeros in the columns of storm ui like emitted and transferred ,
> result
> > file is empty
> >
> > On Wednesday, January 27, 2016, Erik Weathers
> > <ew...@groupon.com.invalid>
> > wrote:
> >
> > > Your mail client is wrapping the log lines prematurely, I have a really
> > > really hard time reading wrapped lines, I'd look into fixing that if I
> > were
> > > you. Here they are unwrapped:
> > >
> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> connection,
> > > connectString=localhost:2181 sessionTimeout=20000
> > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
> to
> > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
> using
> > > SASL (unknown error)
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> established
> > > to localhost/127.0.1.1:2181, initiating session
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > complete
> > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> > > negotiated timeout = 20000
> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> > > :connected:none
> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> > > closed
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> connection,
> > > connectString=localhost:2181/storm sessionTimeout=20000
> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
> to
> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> > authenticate
> > > using SASL (unknown error)
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> established
> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > complete
> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > > 0x152804f3a3a0003,
> > > negotiated timeout = 20000
> > >
> > > None of those indicate a problem, they look pretty standard to me.
> > >
> > > Please spend a bit more time zeroing in on what the actual problem is
> so
> > > that the members of the list(s) can provide help.
> > >
> > > - Erik
> > >
> > > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
> > prog.researcher@gmail.com
> > > <javascript:;>>
> > > wrote:
> > >
> > > > yes i tried in local and worked well
> > > > and about /etc/hosts . i'm feeling that this file has a mistake , i
> > made
> > > > alot of changes in this file and didn't remember what was default
> > > >
> > > > In nimbus log file when it connected zookeeper i got
> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> > connection,
> > > > connectString=localhost:2181 sessionTimeout=20000
> > > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
> > to
> > > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
> > using
> > > > SASL (unknown error)
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> > established
> > > > to localhost/127.0.1.1:2181, initiating session
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > > complete
> > > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> > > > negotiated timeout = 20000
> > > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> > > > :connected:none
> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> > > > closed
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> > > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> > connection,
> > > > connectString=localhost:2181/storm sessionTimeout=20000
> > > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
> > to
> > > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> > > authenticate
> > > > using SASL (unknown error)
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> > established
> > > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > > complete
> > > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > > > 0x152804f3a3a0003,
> > > > negotiated timeout = 20000
> > > >
> > > > and i set for drpc.server : localhost
> > > > storm.zookeeper.server : localhost
> > > > nimbus.host : localhost
> > > >
> > > > in my /etc/hosts
> > > > 127.0.0.1 user-Lenovo-G50-70 localhost
> > > > 127.0.1.1 localhost
> > > >
> > > > is that right ?
> > > >
> > > >
> > > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
> > > > eweathers@groupon.com.invalid> wrote:
> > > >
> > > > > You said: "except the statement of drpc server trying to connect"
> > > > >
> > > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
> > > Distributed
> > > > > RPC servers..." implies?
> > > > > That is just saying that the server is being started. It's a
> > server,
> > > > not
> > > > > a client, so it's basic operation is *not* to connect to some other
> > > > thing.
> > > > > It's up and waiting for you to tell it to do stuff.
> > > > >
> > > > > Have you gotten the Local-mode version of DRPC working?
> > > > >
> > > > > -
> > > > >
> > > > >
> > > >
> > >
> >
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
> > > > >
> > > > > Not sure what you're asking with regards to DNS and /etc/hosts,
> those
> > > > seem
> > > > > unrelated to your basic issue.
> > > > >
> > > > > - Erik
> > > > >
> > > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
> > > > prog.researcher@gmail.com <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > There is no error except the statement of drpc server trying to
> > > connect
> > > > > . I
> > > > > > guess I have problem with Dns . if you have any idea about this
> > > pleaes
> > > > > help
> > > > > >
> > > > > >
> > > > > > I want to submit topology with single machine
> > > > > > Only on my laptop without any other devices as a first step so
> > > > > > What this file /etc/hosts should contains ?
> > > > > > As I set in drpc.server : localhost
> > > > > > Storm.zookeeper. server: localhost
> > > > > > Nimbus.host:localhost
> > > > > >
> > > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
> > > > > >
> > > > > > What should I use and what should I hash it to not using it ?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wednesday, January 27, 2016, Erik Weathers
> > > > > > <ew...@groupon.com.invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > What does the client code that is supposed to make the DRPC
> > > > connection
> > > > > > > telling you? i.e., you should see some exception or log about
> > not
> > > > > > > establishing the connection, right?
> > > > > > >
> > > > > > > Alternatively, perhaps the connections aren't persistent and
> > > there's
> > > > no
> > > > > > > actual problem?
> > > > > > >
> > > > > > > - Erik
> > > > > > >
> > > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > > > > > prog.researcher@gmail.com <javascript:;>
> > > > > > > <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > thanks for replying , i read the documentation before , i
> > > imported
> > > > > > > project
> > > > > > > > supposed to work well but not working with me
> > > > > > > > i checked port by lsof -i gave me all ports i connected it
> for
> > > > storm
> > > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP
> *:52022
> > > > > > (LISTEN)
> > > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP
> *:2181
> > > > > (LISTEN)
> > > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
> > > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
> > > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526
> (ESTABLISHED)
> > > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
> > > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
> > > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP
> *:6627
> > > > > (LISTEN)
> > > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
> > > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181
> (ESTABLISHED)
> > > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP
> *:3772
> > > > > (LISTEN)
> > > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP
> *:3773
> > > > > (LISTEN)
> > > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
> > > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825
> (ESTABLISHED)
> > > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
> > > *:http-alt
> > > > > > > (LISTEN)
> > > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
> > > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773
> (ESTABLISHED)
> > > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
> > > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP
> *:7660
> > > > > (LISTEN)
> > > > > > > >
> > > > > > > >
> > > > > > > > here you can see that 3772 not established
> > > > > > > >
> > > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > > > > > > > eweathers@groupon.com.invalid> wrote:
> > > > > > > >
> > > > > > > > > hey,
> > > > > > > > >
> > > > > > > > > The DRPC server is up and listening on port 3772. Why do
> > you
> > > > > expect
> > > > > > > > > established connections?
> > > > > > > > >
> > > > > > > > > I'm not familiar with using Storm's DRPC feature, but I'm
> > sure
> > > > you
> > > > > > need
> > > > > > > > to
> > > > > > > > > write code that interacts with the DRPC server, and you've
> > made
> > > > no
> > > > > > > > mention
> > > > > > > > > of doing so in your email. I'd start here:
> > > > > > > > >
> > > > > > > > > -
> > > https://storm.apache.org/documentation/Distributed-RPC.html
> > > > > > > > >
> > > > > > > > > - Erik
> > > > > > > > >
> > > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > > > > > > > prog.researcher@gmail.com <javascript:;> <javascript:;>>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I set in the code
> > > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > > > > > > > conf.put(Config.DRPC_PORT, 3772);
> > > > > > > > > > but when i submit topolgoy i found at the end of the file
> > > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > > > > > > > >
> > > > > > > > > > i checked port 3772 by
> > > > > > > > > >
> > > > > > > > > > sudo netstat -ap | grep 3772
> > > > > > > > > >
> > > > > > > > > > i got
> > > > > > > > > >
> > > > > > > > > > tcp 0 0 *:3772 *:* LISTEN
> 10864/java
> > > > > > > > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > why it's not established ?
> > > > > > > > > >
> > > > > > > > > > can i find help ?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
Please put more effort into describing the issue. "It doesn't work" is
unfortunately not enough info for anyone to provide help.
e.g., post links to some code you are trying to run, and the configs of the
storm components that you are running.
- Erik
On Wed, Jan 27, 2016 at 4:59 AM, sam mohel <sa...@gmail.com> wrote:
> I wrote the actual problem in My first message drpc server not working I
> hot zeros in the columns of storm ui like emitted and transferred , result
> file is empty
>
> On Wednesday, January 27, 2016, Erik Weathers
> <ew...@groupon.com.invalid>
> wrote:
>
> > Your mail client is wrapping the log lines prematurely, I have a really
> > really hard time reading wrapped lines, I'd look into fixing that if I
> were
> > you. Here they are unwrapped:
> >
> > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> > connectString=localhost:2181 sessionTimeout=20000
> > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> > server localhost/127.0.1.1:2181. Will not attempt to authenticate using
> > SASL (unknown error)
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> > to localhost/127.0.1.1:2181, initiating session
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> complete
> > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> > negotiated timeout = 20000
> > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> > :connected:none
> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> > closed
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> > connectString=localhost:2181/storm sessionTimeout=20000
> > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> authenticate
> > using SASL (unknown error)
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> complete
> > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > 0x152804f3a3a0003,
> > negotiated timeout = 20000
> >
> > None of those indicate a problem, they look pretty standard to me.
> >
> > Please spend a bit more time zeroing in on what the actual problem is so
> > that the members of the list(s) can provide help.
> >
> > - Erik
> >
> > On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <
> prog.researcher@gmail.com
> > <javascript:;>>
> > wrote:
> >
> > > yes i tried in local and worked well
> > > and about /etc/hosts . i'm feeling that this file has a mistake , i
> made
> > > alot of changes in this file and didn't remember what was default
> > >
> > > In nimbus log file when it connected zookeeper i got
> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> connection,
> > > connectString=localhost:2181 sessionTimeout=20000
> > > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
> to
> > > server localhost/127.0.1.1:2181. Will not attempt to authenticate
> using
> > > SASL (unknown error)
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> established
> > > to localhost/127.0.1.1:2181, initiating session
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > complete
> > > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> > > negotiated timeout = 20000
> > > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> > > :connected:none
> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> > > closed
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> > > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client
> connection,
> > > connectString=localhost:2181/storm sessionTimeout=20000
> > > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection
> to
> > > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> > authenticate
> > > using SASL (unknown error)
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection
> established
> > > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> > complete
> > > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > > 0x152804f3a3a0003,
> > > negotiated timeout = 20000
> > >
> > > and i set for drpc.server : localhost
> > > storm.zookeeper.server : localhost
> > > nimbus.host : localhost
> > >
> > > in my /etc/hosts
> > > 127.0.0.1 user-Lenovo-G50-70 localhost
> > > 127.0.1.1 localhost
> > >
> > > is that right ?
> > >
> > >
> > > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
> > > eweathers@groupon.com.invalid> wrote:
> > >
> > > > You said: "except the statement of drpc server trying to connect"
> > > >
> > > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
> > Distributed
> > > > RPC servers..." implies?
> > > > That is just saying that the server is being started. It's a
> server,
> > > not
> > > > a client, so it's basic operation is *not* to connect to some other
> > > thing.
> > > > It's up and waiting for you to tell it to do stuff.
> > > >
> > > > Have you gotten the Local-mode version of DRPC working?
> > > >
> > > > -
> > > >
> > > >
> > >
> >
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
> > > >
> > > > Not sure what you're asking with regards to DNS and /etc/hosts, those
> > > seem
> > > > unrelated to your basic issue.
> > > >
> > > > - Erik
> > > >
> > > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
> > > prog.researcher@gmail.com <javascript:;>>
> > > > wrote:
> > > >
> > > > > There is no error except the statement of drpc server trying to
> > connect
> > > > . I
> > > > > guess I have problem with Dns . if you have any idea about this
> > pleaes
> > > > help
> > > > >
> > > > >
> > > > > I want to submit topology with single machine
> > > > > Only on my laptop without any other devices as a first step so
> > > > > What this file /etc/hosts should contains ?
> > > > > As I set in drpc.server : localhost
> > > > > Storm.zookeeper. server: localhost
> > > > > Nimbus.host:localhost
> > > > >
> > > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
> > > > >
> > > > > What should I use and what should I hash it to not using it ?
> > > > >
> > > > >
> > > > >
> > > > > On Wednesday, January 27, 2016, Erik Weathers
> > > > > <ew...@groupon.com.invalid>
> > > > > wrote:
> > > > >
> > > > > > What does the client code that is supposed to make the DRPC
> > > connection
> > > > > > telling you? i.e., you should see some exception or log about
> not
> > > > > > establishing the connection, right?
> > > > > >
> > > > > > Alternatively, perhaps the connections aren't persistent and
> > there's
> > > no
> > > > > > actual problem?
> > > > > >
> > > > > > - Erik
> > > > > >
> > > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > > > > prog.researcher@gmail.com <javascript:;>
> > > > > > <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > thanks for replying , i read the documentation before , i
> > imported
> > > > > > project
> > > > > > > supposed to work well but not working with me
> > > > > > > i checked port by lsof -i gave me all ports i connected it for
> > > storm
> > > > > > > java 10675 root 20u IPv4 98126 0t0 TCP *:52022
> > > > > (LISTEN)
> > > > > > > java 10675 root 26u IPv4 98131 0t0 TCP *:2181
> > > > (LISTEN)
> > > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
> > > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
> > > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
> > > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
> > > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > > > > > > java 10715 root 91u IPv4 98245 0t0 TCP *:6627
> > > > (LISTEN)
> > > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
> > > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > > > > > > java 10864 root 82u IPv4 102425 0t0 TCP *:3772
> > > > (LISTEN)
> > > > > > > java 10864 root 84u IPv4 102429 0t0 TCP *:3773
> > > > (LISTEN)
> > > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
> > > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
> > *:http-alt
> > > > > > (LISTEN)
> > > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
> > > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
> > > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > > > > > > java 11087 root 94u IPv4 102561 0t0 TCP *:7660
> > > > (LISTEN)
> > > > > > >
> > > > > > >
> > > > > > > here you can see that 3772 not established
> > > > > > >
> > > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > > > > > > eweathers@groupon.com.invalid> wrote:
> > > > > > >
> > > > > > > > hey,
> > > > > > > >
> > > > > > > > The DRPC server is up and listening on port 3772. Why do
> you
> > > > expect
> > > > > > > > established connections?
> > > > > > > >
> > > > > > > > I'm not familiar with using Storm's DRPC feature, but I'm
> sure
> > > you
> > > > > need
> > > > > > > to
> > > > > > > > write code that interacts with the DRPC server, and you've
> made
> > > no
> > > > > > > mention
> > > > > > > > of doing so in your email. I'd start here:
> > > > > > > >
> > > > > > > > -
> > https://storm.apache.org/documentation/Distributed-RPC.html
> > > > > > > >
> > > > > > > > - Erik
> > > > > > > >
> > > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > > > > > > prog.researcher@gmail.com <javascript:;> <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I set in the code
> > > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > > > > > > conf.put(Config.DRPC_PORT, 3772);
> > > > > > > > > but when i submit topolgoy i found at the end of the file
> > > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > > > > > > >
> > > > > > > > > i checked port 3772 by
> > > > > > > > >
> > > > > > > > > sudo netstat -ap | grep 3772
> > > > > > > > >
> > > > > > > > > i got
> > > > > > > > >
> > > > > > > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > > > > > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > why it's not established ?
> > > > > > > > >
> > > > > > > > > can i find help ?
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by sam mohel <sa...@gmail.com>.
I wrote the actual problem in My first message drpc server not working I
hot zeros in the columns of storm ui like emitted and transferred , result
file is empty
On Wednesday, January 27, 2016, Erik Weathers <ew...@groupon.com.invalid>
wrote:
> Your mail client is wrapping the log lines prematurely, I have a really
> really hard time reading wrapped lines, I'd look into fixing that if I were
> you. Here they are unwrapped:
>
> 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> connectString=localhost:2181 sessionTimeout=20000
> watcher=com.netflix.curator.ConnectionState@2fa423d2
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> server localhost/127.0.1.1:2181. Will not attempt to authenticate using
> SASL (unknown error)
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> to localhost/127.0.1.1:2181, initiating session
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
> on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> negotiated timeout = 20000
> 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> :connected:none
> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> closed
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> connectString=localhost:2181/storm sessionTimeout=20000
> watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
> on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> 0x152804f3a3a0003,
> negotiated timeout = 20000
>
> None of those indicate a problem, they look pretty standard to me.
>
> Please spend a bit more time zeroing in on what the actual problem is so
> that the members of the list(s) can provide help.
>
> - Erik
>
> On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <prog.researcher@gmail.com
> <javascript:;>>
> wrote:
>
> > yes i tried in local and worked well
> > and about /etc/hosts . i'm feeling that this file has a mistake , i made
> > alot of changes in this file and didn't remember what was default
> >
> > In nimbus log file when it connected zookeeper i got
> > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> > connectString=localhost:2181 sessionTimeout=20000
> > watcher=com.netflix.curator.ConnectionState@2fa423d2
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> > server localhost/127.0.1.1:2181. Will not attempt to authenticate using
> > SASL (unknown error)
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> > to localhost/127.0.1.1:2181, initiating session
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> complete
> > on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> > negotiated timeout = 20000
> > 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> > :connected:none
> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> > closed
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> > 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> > 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> > connectString=localhost:2181/storm sessionTimeout=20000
> > watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> > server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to
> authenticate
> > using SASL (unknown error)
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> > to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> > 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment
> complete
> > on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> > 0x152804f3a3a0003,
> > negotiated timeout = 20000
> >
> > and i set for drpc.server : localhost
> > storm.zookeeper.server : localhost
> > nimbus.host : localhost
> >
> > in my /etc/hosts
> > 127.0.0.1 user-Lenovo-G50-70 localhost
> > 127.0.1.1 localhost
> >
> > is that right ?
> >
> >
> > On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
> > eweathers@groupon.com.invalid> wrote:
> >
> > > You said: "except the statement of drpc server trying to connect"
> > >
> > > Maybe you are confused about what "b.s.d.drpc [INFO] Starting
> Distributed
> > > RPC servers..." implies?
> > > That is just saying that the server is being started. It's a server,
> > not
> > > a client, so it's basic operation is *not* to connect to some other
> > thing.
> > > It's up and waiting for you to tell it to do stuff.
> > >
> > > Have you gotten the Local-mode version of DRPC working?
> > >
> > > -
> > >
> > >
> >
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
> > >
> > > Not sure what you're asking with regards to DNS and /etc/hosts, those
> > seem
> > > unrelated to your basic issue.
> > >
> > > - Erik
> > >
> > > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
> > prog.researcher@gmail.com <javascript:;>>
> > > wrote:
> > >
> > > > There is no error except the statement of drpc server trying to
> connect
> > > . I
> > > > guess I have problem with Dns . if you have any idea about this
> pleaes
> > > help
> > > >
> > > >
> > > > I want to submit topology with single machine
> > > > Only on my laptop without any other devices as a first step so
> > > > What this file /etc/hosts should contains ?
> > > > As I set in drpc.server : localhost
> > > > Storm.zookeeper. server: localhost
> > > > Nimbus.host:localhost
> > > >
> > > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
> > > >
> > > > What should I use and what should I hash it to not using it ?
> > > >
> > > >
> > > >
> > > > On Wednesday, January 27, 2016, Erik Weathers
> > > > <ew...@groupon.com.invalid>
> > > > wrote:
> > > >
> > > > > What does the client code that is supposed to make the DRPC
> > connection
> > > > > telling you? i.e., you should see some exception or log about not
> > > > > establishing the connection, right?
> > > > >
> > > > > Alternatively, perhaps the connections aren't persistent and
> there's
> > no
> > > > > actual problem?
> > > > >
> > > > > - Erik
> > > > >
> > > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > > > prog.researcher@gmail.com <javascript:;>
> > > > > <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > thanks for replying , i read the documentation before , i
> imported
> > > > > project
> > > > > > supposed to work well but not working with me
> > > > > > i checked port by lsof -i gave me all ports i connected it for
> > storm
> > > > > > java 10675 root 20u IPv4 98126 0t0 TCP *:52022
> > > > (LISTEN)
> > > > > > java 10675 root 26u IPv4 98131 0t0 TCP *:2181
> > > (LISTEN)
> > > > > > java 10675 root 27u IPv4 101944 0t0 TCP
> > > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > > > > > java 10675 root 29u IPv4 98974 0t0 TCP
> > > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > > > > > java 10675 root 30u IPv4 99105 0t0 TCP
> > > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > > > > > java 10715 root 90u IPv4 98953 0t0 TCP
> > > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > > > > > java 10715 root 91u IPv4 98245 0t0 TCP *:6627
> > > (LISTEN)
> > > > > > java 10792 root 90u IPv4 99973 0t0 TCP
> > > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > > > > > java 10864 root 82u IPv4 102425 0t0 TCP *:3772
> > > (LISTEN)
> > > > > > java 10864 root 84u IPv4 102429 0t0 TCP *:3773
> > > (LISTEN)
> > > > > > java 10864 root 92u IPv4 102197 0t0 TCP
> > > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > > > > > java 10928 root 81u IPv4 102070 0t0 TCP
> *:http-alt
> > > > > (LISTEN)
> > > > > > java 11087 root 81u IPv4 100091 0t0 TCP
> > > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > > > > > java 11087 root 91u IPv4 102196 0t0 TCP
> > > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > > > > > java 11087 root 94u IPv4 102561 0t0 TCP *:7660
> > > (LISTEN)
> > > > > >
> > > > > >
> > > > > > here you can see that 3772 not established
> > > > > >
> > > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > > > > > eweathers@groupon.com.invalid> wrote:
> > > > > >
> > > > > > > hey,
> > > > > > >
> > > > > > > The DRPC server is up and listening on port 3772. Why do you
> > > expect
> > > > > > > established connections?
> > > > > > >
> > > > > > > I'm not familiar with using Storm's DRPC feature, but I'm sure
> > you
> > > > need
> > > > > > to
> > > > > > > write code that interacts with the DRPC server, and you've made
> > no
> > > > > > mention
> > > > > > > of doing so in your email. I'd start here:
> > > > > > >
> > > > > > > -
> https://storm.apache.org/documentation/Distributed-RPC.html
> > > > > > >
> > > > > > > - Erik
> > > > > > >
> > > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > > > > > prog.researcher@gmail.com <javascript:;> <javascript:;>>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I set in the code
> > > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > > > > > conf.put(Config.DRPC_PORT, 3772);
> > > > > > > > but when i submit topolgoy i found at the end of the file
> > > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > > > > > >
> > > > > > > > i checked port 3772 by
> > > > > > > >
> > > > > > > > sudo netstat -ap | grep 3772
> > > > > > > >
> > > > > > > > i got
> > > > > > > >
> > > > > > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > > > > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > > > > > >
> > > > > > > >
> > > > > > > > why it's not established ?
> > > > > > > >
> > > > > > > > can i find help ?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
Your mail client is wrapping the log lines prematurely, I have a really
really hard time reading wrapped lines, I'd look into fixing that if I were
you. Here they are unwrapped:
2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=com.netflix.curator.ConnectionState@2fa423d2
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
server localhost/127.0.1.1:2181. Will not attempt to authenticate using
SASL (unknown error)
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
to localhost/127.0.1.1:2181, initiating session
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
negotiated timeout = 20000
2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
:connected:none
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002 closed
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=com.netflix.curator.ConnectionState@2ee8b0bf
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to authenticate
using SASL (unknown error)
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid = 0x152804f3a3a0003,
negotiated timeout = 20000
None of those indicate a problem, they look pretty standard to me.
Please spend a bit more time zeroing in on what the actual problem is so
that the members of the list(s) can provide help.
- Erik
On Tue, Jan 26, 2016 at 10:15 PM, researcher cs <pr...@gmail.com>
wrote:
> yes i tried in local and worked well
> and about /etc/hosts . i'm feeling that this file has a mistake , i made
> alot of changes in this file and didn't remember what was default
>
> In nimbus log file when it connected zookeeper i got
> 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> connectString=localhost:2181 sessionTimeout=20000
> watcher=com.netflix.curator.ConnectionState@2fa423d2
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> server localhost/127.0.1.1:2181. Will not attempt to authenticate using
> SASL (unknown error)
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> to localhost/127.0.1.1:2181, initiating session
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
> on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
> negotiated timeout = 20000
> 2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
> :connected:none
> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002
> closed
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
> 2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
> connectString=localhost:2181/storm sessionTimeout=20000
> watcher=com.netflix.curator.ConnectionState@2ee8b0bf
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
> server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
> to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
> 2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
> on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid =
> 0x152804f3a3a0003,
> negotiated timeout = 20000
>
> and i set for drpc.server : localhost
> storm.zookeeper.server : localhost
> nimbus.host : localhost
>
> in my /etc/hosts
> 127.0.0.1 user-Lenovo-G50-70 localhost
> 127.0.1.1 localhost
>
> is that right ?
>
>
> On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
> eweathers@groupon.com.invalid> wrote:
>
> > You said: "except the statement of drpc server trying to connect"
> >
> > Maybe you are confused about what "b.s.d.drpc [INFO] Starting Distributed
> > RPC servers..." implies?
> > That is just saying that the server is being started. It's a server,
> not
> > a client, so it's basic operation is *not* to connect to some other
> thing.
> > It's up and waiting for you to tell it to do stuff.
> >
> > Have you gotten the Local-mode version of DRPC working?
> >
> > -
> >
> >
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
> >
> > Not sure what you're asking with regards to DNS and /etc/hosts, those
> seem
> > unrelated to your basic issue.
> >
> > - Erik
> >
> > On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <
> prog.researcher@gmail.com>
> > wrote:
> >
> > > There is no error except the statement of drpc server trying to connect
> > . I
> > > guess I have problem with Dns . if you have any idea about this pleaes
> > help
> > >
> > >
> > > I want to submit topology with single machine
> > > Only on my laptop without any other devices as a first step so
> > > What this file /etc/hosts should contains ?
> > > As I set in drpc.server : localhost
> > > Storm.zookeeper. server: localhost
> > > Nimbus.host:localhost
> > >
> > > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
> > >
> > > What should I use and what should I hash it to not using it ?
> > >
> > >
> > >
> > > On Wednesday, January 27, 2016, Erik Weathers
> > > <ew...@groupon.com.invalid>
> > > wrote:
> > >
> > > > What does the client code that is supposed to make the DRPC
> connection
> > > > telling you? i.e., you should see some exception or log about not
> > > > establishing the connection, right?
> > > >
> > > > Alternatively, perhaps the connections aren't persistent and there's
> no
> > > > actual problem?
> > > >
> > > > - Erik
> > > >
> > > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > > prog.researcher@gmail.com
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > thanks for replying , i read the documentation before , i imported
> > > > project
> > > > > supposed to work well but not working with me
> > > > > i checked port by lsof -i gave me all ports i connected it for
> storm
> > > > > java 10675 root 20u IPv4 98126 0t0 TCP *:52022
> > > (LISTEN)
> > > > > java 10675 root 26u IPv4 98131 0t0 TCP *:2181
> > (LISTEN)
> > > > > java 10675 root 27u IPv4 101944 0t0 TCP
> > > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > > > > java 10675 root 29u IPv4 98974 0t0 TCP
> > > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > > > > java 10675 root 30u IPv4 99105 0t0 TCP
> > > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > > > > java 10715 root 90u IPv4 98953 0t0 TCP
> > > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > > > > java 10715 root 91u IPv4 98245 0t0 TCP *:6627
> > (LISTEN)
> > > > > java 10792 root 90u IPv4 99973 0t0 TCP
> > > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > > > > java 10864 root 82u IPv4 102425 0t0 TCP *:3772
> > (LISTEN)
> > > > > java 10864 root 84u IPv4 102429 0t0 TCP *:3773
> > (LISTEN)
> > > > > java 10864 root 92u IPv4 102197 0t0 TCP
> > > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > > > > java 10928 root 81u IPv4 102070 0t0 TCP *:http-alt
> > > > (LISTEN)
> > > > > java 11087 root 81u IPv4 100091 0t0 TCP
> > > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > > > > java 11087 root 91u IPv4 102196 0t0 TCP
> > > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > > > > java 11087 root 94u IPv4 102561 0t0 TCP *:7660
> > (LISTEN)
> > > > >
> > > > >
> > > > > here you can see that 3772 not established
> > > > >
> > > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > > > > eweathers@groupon.com.invalid> wrote:
> > > > >
> > > > > > hey,
> > > > > >
> > > > > > The DRPC server is up and listening on port 3772. Why do you
> > expect
> > > > > > established connections?
> > > > > >
> > > > > > I'm not familiar with using Storm's DRPC feature, but I'm sure
> you
> > > need
> > > > > to
> > > > > > write code that interacts with the DRPC server, and you've made
> no
> > > > > mention
> > > > > > of doing so in your email. I'd start here:
> > > > > >
> > > > > > - https://storm.apache.org/documentation/Distributed-RPC.html
> > > > > >
> > > > > > - Erik
> > > > > >
> > > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > > > > prog.researcher@gmail.com <javascript:;>>
> > > > > > wrote:
> > > > > >
> > > > > > > I set in the code
> > > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > > > > conf.put(Config.DRPC_PORT, 3772);
> > > > > > > but when i submit topolgoy i found at the end of the file
> > > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > > > > >
> > > > > > > i checked port 3772 by
> > > > > > >
> > > > > > > sudo netstat -ap | grep 3772
> > > > > > >
> > > > > > > i got
> > > > > > >
> > > > > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > > > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > > > > >
> > > > > > >
> > > > > > > why it's not established ?
> > > > > > >
> > > > > > > can i find help ?
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
yes i tried in local and worked well
and about /etc/hosts . i'm feeling that this file has a mistake , i made
alot of changes in this file and didn't remember what was default
In nimbus log file when it connected zookeeper i got
2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=localhost:2181 sessionTimeout=20000
watcher=com.netflix.curator.ConnectionState@2fa423d2
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
server localhost/127.0.1.1:2181. Will not attempt to authenticate using
SASL (unknown error)
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
to localhost/127.0.1.1:2181, initiating session
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
on server localhost/127.0.1.1:2181, sessionid = 0x152804f3a3a0002,
negotiated timeout = 20000
2016-01-27 01:41:00 b.s.zookeeper [INFO] Zookeeper state update:
:connected:none
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Session: 0x152804f3a3a0002 closed
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] EventThread shut down
2016-01-27 01:41:00 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-01-27 01:41:00 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=localhost:2181/storm sessionTimeout=20000
watcher=com.netflix.curator.ConnectionState@2ee8b0bf
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Opening socket connection to
server user-Lenovo-G50-70/127.0.0.1:2181. Will not attempt to authenticate
using SASL (unknown error)
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Socket connection established
to user-Lenovo-G50-70/127.0.0.1:2181, initiating session
2016-01-27 01:41:00 o.a.z.ClientCnxn [INFO] Session establishment complete
on server user-Lenovo-G50-70/127.0.0.1:2181, sessionid = 0x152804f3a3a0003,
negotiated timeout = 20000
and i set for drpc.server : localhost
storm.zookeeper.server : localhost
nimbus.host : localhost
in my /etc/hosts
127.0.0.1 user-Lenovo-G50-70 localhost
127.0.1.1 localhost
is that right ?
On Wed, Jan 27, 2016 at 6:12 AM, Erik Weathers <
eweathers@groupon.com.invalid> wrote:
> You said: "except the statement of drpc server trying to connect"
>
> Maybe you are confused about what "b.s.d.drpc [INFO] Starting Distributed
> RPC servers..." implies?
> That is just saying that the server is being started. It's a server, not
> a client, so it's basic operation is *not* to connect to some other thing.
> It's up and waiting for you to tell it to do stuff.
>
> Have you gotten the Local-mode version of DRPC working?
>
> -
>
> https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
>
> Not sure what you're asking with regards to DNS and /etc/hosts, those seem
> unrelated to your basic issue.
>
> - Erik
>
> On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <pr...@gmail.com>
> wrote:
>
> > There is no error except the statement of drpc server trying to connect
> . I
> > guess I have problem with Dns . if you have any idea about this pleaes
> help
> >
> >
> > I want to submit topology with single machine
> > Only on my laptop without any other devices as a first step so
> > What this file /etc/hosts should contains ?
> > As I set in drpc.server : localhost
> > Storm.zookeeper. server: localhost
> > Nimbus.host:localhost
> >
> > As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
> >
> > What should I use and what should I hash it to not using it ?
> >
> >
> >
> > On Wednesday, January 27, 2016, Erik Weathers
> > <ew...@groupon.com.invalid>
> > wrote:
> >
> > > What does the client code that is supposed to make the DRPC connection
> > > telling you? i.e., you should see some exception or log about not
> > > establishing the connection, right?
> > >
> > > Alternatively, perhaps the connections aren't persistent and there's no
> > > actual problem?
> > >
> > > - Erik
> > >
> > > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> > prog.researcher@gmail.com
> > > <javascript:;>>
> > > wrote:
> > >
> > > > thanks for replying , i read the documentation before , i imported
> > > project
> > > > supposed to work well but not working with me
> > > > i checked port by lsof -i gave me all ports i connected it for storm
> > > > java 10675 root 20u IPv4 98126 0t0 TCP *:52022
> > (LISTEN)
> > > > java 10675 root 26u IPv4 98131 0t0 TCP *:2181
> (LISTEN)
> > > > java 10675 root 27u IPv4 101944 0t0 TCP
> > > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > > > java 10675 root 29u IPv4 98974 0t0 TCP
> > > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > > > java 10675 root 30u IPv4 99105 0t0 TCP
> > > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > > > java 10715 root 90u IPv4 98953 0t0 TCP
> > > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > > > java 10715 root 91u IPv4 98245 0t0 TCP *:6627
> (LISTEN)
> > > > java 10792 root 90u IPv4 99973 0t0 TCP
> > > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > > > java 10864 root 82u IPv4 102425 0t0 TCP *:3772
> (LISTEN)
> > > > java 10864 root 84u IPv4 102429 0t0 TCP *:3773
> (LISTEN)
> > > > java 10864 root 92u IPv4 102197 0t0 TCP
> > > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > > > java 10928 root 81u IPv4 102070 0t0 TCP *:http-alt
> > > (LISTEN)
> > > > java 11087 root 81u IPv4 100091 0t0 TCP
> > > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > > > java 11087 root 91u IPv4 102196 0t0 TCP
> > > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > > > java 11087 root 94u IPv4 102561 0t0 TCP *:7660
> (LISTEN)
> > > >
> > > >
> > > > here you can see that 3772 not established
> > > >
> > > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > > > eweathers@groupon.com.invalid> wrote:
> > > >
> > > > > hey,
> > > > >
> > > > > The DRPC server is up and listening on port 3772. Why do you
> expect
> > > > > established connections?
> > > > >
> > > > > I'm not familiar with using Storm's DRPC feature, but I'm sure you
> > need
> > > > to
> > > > > write code that interacts with the DRPC server, and you've made no
> > > > mention
> > > > > of doing so in your email. I'd start here:
> > > > >
> > > > > - https://storm.apache.org/documentation/Distributed-RPC.html
> > > > >
> > > > > - Erik
> > > > >
> > > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > > > prog.researcher@gmail.com <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > I set in the code
> > > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > > > conf.put(Config.DRPC_PORT, 3772);
> > > > > > but when i submit topolgoy i found at the end of the file
> > > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > > > >
> > > > > > i checked port 3772 by
> > > > > >
> > > > > > sudo netstat -ap | grep 3772
> > > > > >
> > > > > > i got
> > > > > >
> > > > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > > > >
> > > > > >
> > > > > > why it's not established ?
> > > > > >
> > > > > > can i find help ?
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
You said: "except the statement of drpc server trying to connect"
Maybe you are confused about what "b.s.d.drpc [INFO] Starting Distributed
RPC servers..." implies?
That is just saying that the server is being started. It's a server, not
a client, so it's basic operation is *not* to connect to some other thing.
It's up and waiting for you to tell it to do stuff.
Have you gotten the Local-mode version of DRPC working?
-
https://storm.apache.org/documentation/Distributed-RPC.html#local-mode-drpc
Not sure what you're asking with regards to DNS and /etc/hosts, those seem
unrelated to your basic issue.
- Erik
On Tue, Jan 26, 2016 at 6:58 PM, researcher cs <pr...@gmail.com>
wrote:
> There is no error except the statement of drpc server trying to connect . I
> guess I have problem with Dns . if you have any idea about this pleaes help
>
>
> I want to submit topology with single machine
> Only on my laptop without any other devices as a first step so
> What this file /etc/hosts should contains ?
> As I set in drpc.server : localhost
> Storm.zookeeper. server: localhost
> Nimbus.host:localhost
>
> As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
>
> What should I use and what should I hash it to not using it ?
>
>
>
> On Wednesday, January 27, 2016, Erik Weathers
> <ew...@groupon.com.invalid>
> wrote:
>
> > What does the client code that is supposed to make the DRPC connection
> > telling you? i.e., you should see some exception or log about not
> > establishing the connection, right?
> >
> > Alternatively, perhaps the connections aren't persistent and there's no
> > actual problem?
> >
> > - Erik
> >
> > On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <
> prog.researcher@gmail.com
> > <javascript:;>>
> > wrote:
> >
> > > thanks for replying , i read the documentation before , i imported
> > project
> > > supposed to work well but not working with me
> > > i checked port by lsof -i gave me all ports i connected it for storm
> > > java 10675 root 20u IPv4 98126 0t0 TCP *:52022
> (LISTEN)
> > > java 10675 root 26u IPv4 98131 0t0 TCP *:2181 (LISTEN)
> > > java 10675 root 27u IPv4 101944 0t0 TCP
> > > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > > java 10675 root 29u IPv4 98974 0t0 TCP
> > > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > > java 10675 root 30u IPv4 99105 0t0 TCP
> > > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > > java 10715 root 90u IPv4 98953 0t0 TCP
> > > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > > java 10715 root 91u IPv4 98245 0t0 TCP *:6627 (LISTEN)
> > > java 10792 root 90u IPv4 99973 0t0 TCP
> > > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > > java 10864 root 82u IPv4 102425 0t0 TCP *:3772 (LISTEN)
> > > java 10864 root 84u IPv4 102429 0t0 TCP *:3773 (LISTEN)
> > > java 10864 root 92u IPv4 102197 0t0 TCP
> > > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > > java 10928 root 81u IPv4 102070 0t0 TCP *:http-alt
> > (LISTEN)
> > > java 11087 root 81u IPv4 100091 0t0 TCP
> > > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > > java 11087 root 91u IPv4 102196 0t0 TCP
> > > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > > java 11087 root 94u IPv4 102561 0t0 TCP *:7660 (LISTEN)
> > >
> > >
> > > here you can see that 3772 not established
> > >
> > > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > > eweathers@groupon.com.invalid> wrote:
> > >
> > > > hey,
> > > >
> > > > The DRPC server is up and listening on port 3772. Why do you expect
> > > > established connections?
> > > >
> > > > I'm not familiar with using Storm's DRPC feature, but I'm sure you
> need
> > > to
> > > > write code that interacts with the DRPC server, and you've made no
> > > mention
> > > > of doing so in your email. I'd start here:
> > > >
> > > > - https://storm.apache.org/documentation/Distributed-RPC.html
> > > >
> > > > - Erik
> > > >
> > > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > > prog.researcher@gmail.com <javascript:;>>
> > > > wrote:
> > > >
> > > > > I set in the code
> > > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > > conf.put(Config.DRPC_PORT, 3772);
> > > > > but when i submit topolgoy i found at the end of the file
> > > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > > >
> > > > > i checked port 3772 by
> > > > >
> > > > > sudo netstat -ap | grep 3772
> > > > >
> > > > > i got
> > > > >
> > > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > > >
> > > > >
> > > > > why it's not established ?
> > > > >
> > > > > can i find help ?
> > > > >
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
There is no error except the statement of drpc server trying to connect . I
guess I have problem with Dns . if you have any idea about this pleaes help
I want to submit topology with single machine
Only on my laptop without any other devices as a first step so
What this file /etc/hosts should contains ?
As I set in drpc.server : localhost
Storm.zookeeper. server: localhost
Nimbus.host:localhost
As this file contains 127.0.1.1 and 127.0.0.1 and my IP address ?
What should I use and what should I hash it to not using it ?
On Wednesday, January 27, 2016, Erik Weathers <ew...@groupon.com.invalid>
wrote:
> What does the client code that is supposed to make the DRPC connection
> telling you? i.e., you should see some exception or log about not
> establishing the connection, right?
>
> Alternatively, perhaps the connections aren't persistent and there's no
> actual problem?
>
> - Erik
>
> On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <prog.researcher@gmail.com
> <javascript:;>>
> wrote:
>
> > thanks for replying , i read the documentation before , i imported
> project
> > supposed to work well but not working with me
> > i checked port by lsof -i gave me all ports i connected it for storm
> > java 10675 root 20u IPv4 98126 0t0 TCP *:52022 (LISTEN)
> > java 10675 root 26u IPv4 98131 0t0 TCP *:2181 (LISTEN)
> > java 10675 root 27u IPv4 101944 0t0 TCP
> > localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> > java 10675 root 29u IPv4 98974 0t0 TCP
> > user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> > java 10675 root 30u IPv4 99105 0t0 TCP
> > localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> > java 10715 root 90u IPv4 98953 0t0 TCP
> > user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> > java 10715 root 91u IPv4 98245 0t0 TCP *:6627 (LISTEN)
> > java 10792 root 90u IPv4 99973 0t0 TCP
> > user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> > java 10864 root 82u IPv4 102425 0t0 TCP *:3772 (LISTEN)
> > java 10864 root 84u IPv4 102429 0t0 TCP *:3773 (LISTEN)
> > java 10864 root 92u IPv4 102197 0t0 TCP
> > user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> > java 10928 root 81u IPv4 102070 0t0 TCP *:http-alt
> (LISTEN)
> > java 11087 root 81u IPv4 100091 0t0 TCP
> > user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> > java 11087 root 91u IPv4 102196 0t0 TCP
> > user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> > java 11087 root 94u IPv4 102561 0t0 TCP *:7660 (LISTEN)
> >
> >
> > here you can see that 3772 not established
> >
> > On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> > eweathers@groupon.com.invalid> wrote:
> >
> > > hey,
> > >
> > > The DRPC server is up and listening on port 3772. Why do you expect
> > > established connections?
> > >
> > > I'm not familiar with using Storm's DRPC feature, but I'm sure you need
> > to
> > > write code that interacts with the DRPC server, and you've made no
> > mention
> > > of doing so in your email. I'd start here:
> > >
> > > - https://storm.apache.org/documentation/Distributed-RPC.html
> > >
> > > - Erik
> > >
> > > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> > prog.researcher@gmail.com <javascript:;>>
> > > wrote:
> > >
> > > > I set in the code
> > > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > > conf.put(Config.DRPC_PORT, 3772);
> > > > but when i submit topolgoy i found at the end of the file
> > > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > > >
> > > > i checked port 3772 by
> > > >
> > > > sudo netstat -ap | grep 3772
> > > >
> > > > i got
> > > >
> > > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > > unix 3 [ ] STREAM CONNECTED 13772
> > > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > > >
> > > >
> > > > why it's not established ?
> > > >
> > > > can i find help ?
> > > >
> > >
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
What does the client code that is supposed to make the DRPC connection
telling you? i.e., you should see some exception or log about not
establishing the connection, right?
Alternatively, perhaps the connections aren't persistent and there's no
actual problem?
- Erik
On Tue, Jan 26, 2016 at 4:55 PM, researcher cs <pr...@gmail.com>
wrote:
> thanks for replying , i read the documentation before , i imported project
> supposed to work well but not working with me
> i checked port by lsof -i gave me all ports i connected it for storm
> java 10675 root 20u IPv4 98126 0t0 TCP *:52022 (LISTEN)
> java 10675 root 26u IPv4 98131 0t0 TCP *:2181 (LISTEN)
> java 10675 root 27u IPv4 101944 0t0 TCP
> localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
> java 10675 root 29u IPv4 98974 0t0 TCP
> user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
> java 10675 root 30u IPv4 99105 0t0 TCP
> localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
> java 10715 root 90u IPv4 98953 0t0 TCP
> user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
> java 10715 root 91u IPv4 98245 0t0 TCP *:6627 (LISTEN)
> java 10792 root 90u IPv4 99973 0t0 TCP
> user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
> java 10864 root 82u IPv4 102425 0t0 TCP *:3772 (LISTEN)
> java 10864 root 84u IPv4 102429 0t0 TCP *:3773 (LISTEN)
> java 10864 root 92u IPv4 102197 0t0 TCP
> user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
> java 10928 root 81u IPv4 102070 0t0 TCP *:http-alt (LISTEN)
> java 11087 root 81u IPv4 100091 0t0 TCP
> user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
> java 11087 root 91u IPv4 102196 0t0 TCP
> user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
> java 11087 root 94u IPv4 102561 0t0 TCP *:7660 (LISTEN)
>
>
> here you can see that 3772 not established
>
> On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
> eweathers@groupon.com.invalid> wrote:
>
> > hey,
> >
> > The DRPC server is up and listening on port 3772. Why do you expect
> > established connections?
> >
> > I'm not familiar with using Storm's DRPC feature, but I'm sure you need
> to
> > write code that interacts with the DRPC server, and you've made no
> mention
> > of doing so in your email. I'd start here:
> >
> > - https://storm.apache.org/documentation/Distributed-RPC.html
> >
> > - Erik
> >
> > On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <
> prog.researcher@gmail.com>
> > wrote:
> >
> > > I set in the code
> > > conf.put(Config.DRPC_SERVERS, dprcServers);
> > > conf.put(Config.DRPC_PORT, 3772);
> > > but when i submit topolgoy i found at the end of the file
> > > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> > >
> > > i checked port 3772 by
> > >
> > > sudo netstat -ap | grep 3772
> > >
> > > i got
> > >
> > > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > > unix 3 [ ] STREAM CONNECTED 13772
> > > 587/dbus-daemon /var/run/dbus/system_bus_socket
> > >
> > >
> > > why it's not established ?
> > >
> > > can i find help ?
> > >
> >
>
Re: DRPC server not working
Posted by researcher cs <pr...@gmail.com>.
thanks for replying , i read the documentation before , i imported project
supposed to work well but not working with me
i checked port by lsof -i gave me all ports i connected it for storm
java 10675 root 20u IPv4 98126 0t0 TCP *:52022 (LISTEN)
java 10675 root 26u IPv4 98131 0t0 TCP *:2181 (LISTEN)
java 10675 root 27u IPv4 101944 0t0 TCP
localhost:2181->user-Lenovo-G50-70:38150 (ESTABLISHED)
java 10675 root 29u IPv4 98974 0t0 TCP
user-Lenovo-G50-70:2181->user-Lenovo-G50-70:50526 (ESTABLISHED)
java 10675 root 30u IPv4 99105 0t0 TCP
localhost:2181->user-Lenovo-G50-70:38165 (ESTABLISHED)
java 10715 root 90u IPv4 98953 0t0 TCP
user-Lenovo-G50-70:38150->localhost:2181 (ESTABLISHED)
java 10715 root 91u IPv4 98245 0t0 TCP *:6627 (LISTEN)
java 10792 root 90u IPv4 99973 0t0 TCP
user-Lenovo-G50-70:50526->user-Lenovo-G50-70:2181 (ESTABLISHED)
java 10864 root 82u IPv4 102425 0t0 TCP *:3772 (LISTEN)
java 10864 root 84u IPv4 102429 0t0 TCP *:3773 (LISTEN)
java 10864 root 92u IPv4 102197 0t0 TCP
user-Lenovo-G50-70:3773->user-Lenovo-G50-70:50825 (ESTABLISHED)
java 10928 root 81u IPv4 102070 0t0 TCP *:http-alt (LISTEN)
java 11087 root 81u IPv4 100091 0t0 TCP
user-Lenovo-G50-70:50825->user-Lenovo-G50-70:3773 (ESTABLISHED)
java 11087 root 91u IPv4 102196 0t0 TCP
user-Lenovo-G50-70:38165->localhost:2181 (ESTABLISHED)
java 11087 root 94u IPv4 102561 0t0 TCP *:7660 (LISTEN)
here you can see that 3772 not established
On Wed, Jan 27, 2016 at 2:47 AM, Erik Weathers <
eweathers@groupon.com.invalid> wrote:
> hey,
>
> The DRPC server is up and listening on port 3772. Why do you expect
> established connections?
>
> I'm not familiar with using Storm's DRPC feature, but I'm sure you need to
> write code that interacts with the DRPC server, and you've made no mention
> of doing so in your email. I'd start here:
>
> - https://storm.apache.org/documentation/Distributed-RPC.html
>
> - Erik
>
> On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <pr...@gmail.com>
> wrote:
>
> > I set in the code
> > conf.put(Config.DRPC_SERVERS, dprcServers);
> > conf.put(Config.DRPC_PORT, 3772);
> > but when i submit topolgoy i found at the end of the file
> > b.s.d.drpc [INFO] Starting Distributed RPC servers...
> >
> > i checked port 3772 by
> >
> > sudo netstat -ap | grep 3772
> >
> > i got
> >
> > tcp 0 0 *:3772 *:* LISTEN 10864/java
> > unix 3 [ ] STREAM CONNECTED 13772
> > 587/dbus-daemon /var/run/dbus/system_bus_socket
> >
> >
> > why it's not established ?
> >
> > can i find help ?
> >
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com>.
hey,
The DRPC server is up and listening on port 3772. Why do you expect
established connections?
I'm not familiar with using Storm's DRPC feature, but I'm sure you need to
write code that interacts with the DRPC server, and you've made no mention
of doing so in your email. I'd start here:
- https://storm.apache.org/documentation/Distributed-RPC.html
- Erik
On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <pr...@gmail.com>
wrote:
> I set in the code
> conf.put(Config.DRPC_SERVERS, dprcServers);
> conf.put(Config.DRPC_PORT, 3772);
> but when i submit topolgoy i found at the end of the file
> b.s.d.drpc [INFO] Starting Distributed RPC servers...
>
> i checked port 3772 by
>
> sudo netstat -ap | grep 3772
>
> i got
>
> tcp 0 0 *:3772 *:* LISTEN 10864/java
> unix 3 [ ] STREAM CONNECTED 13772
> 587/dbus-daemon /var/run/dbus/system_bus_socket
>
>
> why it's not established ?
>
> can i find help ?
>
Re: DRPC server not working
Posted by Erik Weathers <ew...@groupon.com.INVALID>.
hey,
The DRPC server is up and listening on port 3772. Why do you expect
established connections?
I'm not familiar with using Storm's DRPC feature, but I'm sure you need to
write code that interacts with the DRPC server, and you've made no mention
of doing so in your email. I'd start here:
- https://storm.apache.org/documentation/Distributed-RPC.html
- Erik
On Tue, Jan 26, 2016 at 4:29 PM, researcher cs <pr...@gmail.com>
wrote:
> I set in the code
> conf.put(Config.DRPC_SERVERS, dprcServers);
> conf.put(Config.DRPC_PORT, 3772);
> but when i submit topolgoy i found at the end of the file
> b.s.d.drpc [INFO] Starting Distributed RPC servers...
>
> i checked port 3772 by
>
> sudo netstat -ap | grep 3772
>
> i got
>
> tcp 0 0 *:3772 *:* LISTEN 10864/java
> unix 3 [ ] STREAM CONNECTED 13772
> 587/dbus-daemon /var/run/dbus/system_bus_socket
>
>
> why it's not established ?
>
> can i find help ?
>