RE: Writing file to HDFS from Java

["Tinte garcia, Miguel Angel" <mi...@atos.net>]

Ok, I am using Flume embedded agent to insert into HBase instead of HDFS. I am using this configuration:
              properties.put("channel.type", "memory");
              properties.put("channel.capacity", "100000");
              properties.put("channel.transactionCapacity", "1000");
              properties.put("sinks", "hbaseSink");
              properties.put("hbaseSink.hostname", "localhost");
              properties.put("hbaseSink.port", port);
              properties.put("hbaseSink.type", "avro"); // only avro is supported
              properties.put("hbaseSink.channel","memoryChannel");
              properties.put("hbaseSink.table", tableName);
              properties.put("hbaseSink.columnFamily", columnFamily);
              properties.put("hbaseSink.serializer","org.apache.flume.sink.hbase.RegexHbaseEventSerializer");
              properties.put("processor.type", "failover");

Apparently, embedded agent is created correctly:
              try {
                     agent = new EmbeddedAgent("agentHBase");
                     agent.configure(properties);
                     agent.start();
              } catch (final Exception ex) {
                     System.err.println(ex.toString());
                     }
              }

Once it is running, I am trying to connect it through:
client = RpcClientFactory.getDefaultInstance(hostname, port, 10000);

But it is returning a connection error:
“org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 61616 }: RPC connection error”

Any idea how can I connect to embedded agent?

Thanks

From: Hari Shreedharan [mailto:hshreedharan@cloudera.com]
Sent: Tuesday, November 11, 2014 7:44 PM
To: user@flume.apache.org
Cc: user@flume.apache.org
Subject: Re: Writing file to HDFS from Java

Are you talking about the embedded agent? Embedded agent is meant to only write data to another Flume agent and does not support the HDFS sink. Only Avro Sink is supported.

Thanks,
Hari


On Tue, Nov 11, 2014 at 10:10 AM, Tinte garcia, Miguel Angel <mi...@atos.net>> wrote:

This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavors to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.

Este mensaje y los ficheros adjuntos pueden contener información confidencial destinada solamente a la(s) persona(s) mencionadas anteriormente y pueden estar protegidos por secreto profesional.
Si usted recibe este correo electrónico por error, gracias por informar inmediatamente al remitente y destruir el mensaje.
Al no estar asegurada la integridad de este mensaje sobre la red, Atos no se hace responsable por su contenido. Su contenido no constituye ningún compromiso para el grupo Atos, salvo ratificación escrita por ambas partes.
Aunque se esfuerza al máximo por mantener su red libre de virus, el emisor no puede garantizar nada al respecto y no será responsable de cualesquiera daños que puedan resultar de una transmisión de virus.