You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Shashi Vishwakarma <sh...@gmail.com> on 2015/12/06 15:53:35 UTC
Flume | Curl Post is not sending data to flume using http source
HI All
down votefavorite
<http://stackoverflow.com/questions/34118713/flume-curl-post-is-not-sending-data-to-flume-using-http-source#>
I have written flume script for receiving data from http source,but flume
is not receiving data from curl post command. Here is my script
########## NEW AGENT ##########
# flume-ng agent -f /etc/flume/conf/flume.httptest.conf -n httpagent
#
# slagent = SysLogAgent
###############################
httpagent.sources = http-source
httpagent.sinks = local-file-sink
httpagent.channels = ch3
# Define / Configure Source (multiport seems to support newer "stuff")
###############################
httpagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
httpagent.sources.http-source.channels = ch3
httpagent.sources.http-source.port = 8181
httpagent.sources.http-source.handler = org.apache.flume.source.http.JSONHandler
httpagent.sources.http-source.bind = localhost
# Local File Sink
###############################
httpagent.sinks.local-file-sink.type = file_roll
httpagent.sinks.local-file-sink.channel = ch3
httpagent.sinks.local-file-sink.sink.directory = /root/Desktop/http_test
httpagent.sinks.local-file-sink.rollInterval = 5
# Channels
###############################
httpagent.channels.ch3.type = memory
httpagent.channels.ch3.capacity = 1000
here is my curl post command
curl -H "Content-Type: application/json" -X POST -d
'[{"username":"shashi","password":"xy1z"}]' http://localhost:8181
After firing this command nothing happens. I dont see any logging in flume.
Even i tried "nc localhost 8181" as well. nc command is also not getting
anything.
Any clue on this?
Re: Flume | Curl Post is not sending data to flume using http source
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Thanks a lot. It worked.....That is really great help....
On Mon, Dec 7, 2015 at 1:01 AM, Gonzalo Herreros <gh...@gmail.com>
wrote:
> I think the problem is in your json, while you are sending an array on
> events, the event doesn't match what Flume expects which is the properties
> headers (optional) and body (string).
> Try like this:
>
> curl -H "Content-Type: application/json" -X POST -d '[{"body": "{\"username\":\"shashi\",\"password\":\"xy1z\"}"}]' http://localhost:8181
>
> Notice the body is a json string, not a json object.
> Your current request should be getting a 400 bad request response code,
> otherwise it would be incorrect.
>
> Regards,
> Gonzalo
>
>
>
> On 6 December 2015 at 14:53, Shashi Vishwakarma <sh...@gmail.com>
> wrote:
>
>> HI All
>>
>>
>> down votefavorite
>> <http://stackoverflow.com/questions/34118713/flume-curl-post-is-not-sending-data-to-flume-using-http-source#>
>>
>> I have written flume script for receiving data from http source,but flume
>> is not receiving data from curl post command. Here is my script
>>
>> ########## NEW AGENT ##########
>> # flume-ng agent -f /etc/flume/conf/flume.httptest.conf -n httpagent
>> #
>>
>> # slagent = SysLogAgent
>> ###############################
>> httpagent.sources = http-source
>> httpagent.sinks = local-file-sink
>> httpagent.channels = ch3
>>
>> # Define / Configure Source (multiport seems to support newer "stuff")
>> ###############################
>> httpagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
>> httpagent.sources.http-source.channels = ch3
>> httpagent.sources.http-source.port = 8181
>> httpagent.sources.http-source.handler = org.apache.flume.source.http.JSONHandler
>> httpagent.sources.http-source.bind = localhost
>>
>>
>> # Local File Sink
>> ###############################
>> httpagent.sinks.local-file-sink.type = file_roll
>> httpagent.sinks.local-file-sink.channel = ch3
>> httpagent.sinks.local-file-sink.sink.directory = /root/Desktop/http_test
>> httpagent.sinks.local-file-sink.rollInterval = 5
>>
>> # Channels
>> ###############################
>> httpagent.channels.ch3.type = memory
>> httpagent.channels.ch3.capacity = 1000
>>
>> here is my curl post command
>>
>> curl -H "Content-Type: application/json" -X POST -d '[{"username":"shashi","password":"xy1z"}]' http://localhost:8181
>>
>> After firing this command nothing happens. I dont see any logging in
>> flume.
>>
>> Even i tried "nc localhost 8181" as well. nc command is also not getting
>> anything.
>>
>> Any clue on this?
>>
>
>
Re: Flume | Curl Post is not sending data to flume using http source
Posted by Gonzalo Herreros <gh...@gmail.com>.
I think the problem is in your json, while you are sending an array on
events, the event doesn't match what Flume expects which is the properties
headers (optional) and body (string).
Try like this:
curl -H "Content-Type: application/json" -X POST -d '[{"body":
"{\"username\":\"shashi\",\"password\":\"xy1z\"}"}]'
http://localhost:8181
Notice the body is a json string, not a json object.
Your current request should be getting a 400 bad request response code,
otherwise it would be incorrect.
Regards,
Gonzalo
On 6 December 2015 at 14:53, Shashi Vishwakarma <sh...@gmail.com>
wrote:
> HI All
>
>
> down votefavorite
> <http://stackoverflow.com/questions/34118713/flume-curl-post-is-not-sending-data-to-flume-using-http-source#>
>
> I have written flume script for receiving data from http source,but flume
> is not receiving data from curl post command. Here is my script
>
> ########## NEW AGENT ##########
> # flume-ng agent -f /etc/flume/conf/flume.httptest.conf -n httpagent
> #
>
> # slagent = SysLogAgent
> ###############################
> httpagent.sources = http-source
> httpagent.sinks = local-file-sink
> httpagent.channels = ch3
>
> # Define / Configure Source (multiport seems to support newer "stuff")
> ###############################
> httpagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
> httpagent.sources.http-source.channels = ch3
> httpagent.sources.http-source.port = 8181
> httpagent.sources.http-source.handler = org.apache.flume.source.http.JSONHandler
> httpagent.sources.http-source.bind = localhost
>
>
> # Local File Sink
> ###############################
> httpagent.sinks.local-file-sink.type = file_roll
> httpagent.sinks.local-file-sink.channel = ch3
> httpagent.sinks.local-file-sink.sink.directory = /root/Desktop/http_test
> httpagent.sinks.local-file-sink.rollInterval = 5
>
> # Channels
> ###############################
> httpagent.channels.ch3.type = memory
> httpagent.channels.ch3.capacity = 1000
>
> here is my curl post command
>
> curl -H "Content-Type: application/json" -X POST -d '[{"username":"shashi","password":"xy1z"}]' http://localhost:8181
>
> After firing this command nothing happens. I dont see any logging in flume.
>
> Even i tried "nc localhost 8181" as well. nc command is also not getting
> anything.
>
> Any clue on this?
>