You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by sam mohel <sa...@gmail.com> on 2017/01/21 14:11:15 UTC
open file in spout with python
I'm new to python , i'm trying to make spout read from file line by line .
i tried to write this statments but didn't work
import storm
import random
# Define some sentences
class SimSpout(storm.Spout):
# Not much to do here for such a basic spout
def initialize(self, conf, context):
## Open the file with read only permit
f = open('mydata.txt', 'r')
self._conf = conf
self._context = context
storm.logInfo("Spout instance starting...")
# Process the next tuple
def nextTuple(self):
# check if it reach at the EOF to close it
line = f.readline()
while line != "":
# Emit a random sentence
storm.logInfo("Emiting %s" % line)
storm.emit([line])
# Start the spout when it's invoked
SimSpout().run()
Another thing i want to ask about , i have shell script that take my data
text file to make process on it to get the result , now if i want to make
it in storm i should make spout open my data text file as its my data
source and read line by line to take every line it got it to bolt that make
a process on it which will contain the code in shell written in python ,
Right ?
Re: open file in spout with python
Posted by Paul Poulosky <pp...@yahoo-inc.com>.
It's hard to know how to debug this without instrumentation. Do you know what's failing? Are you bundling the file you want to read with your topology, otherwise it will have be installed on every node running a supervisor.
On Saturday, January 21, 2017 10:41 PM, sam mohel <sa...@gmail.com> wrote:
Is there any help ?
On Saturday, January 21, 2017, sam mohel <sa...@gmail.com> wrote:
I'm new to python , i'm trying to make spout read from file line by line . i tried to write this statments but didn't work
import storm
import random
# Define some sentences
class SimSpout(storm.Spout):
# Not much to do here for such a basic spout
def initialize(self, conf, context):
## Open the file with read only permit
f = open('mydata.txt', 'r')
self._conf = conf
self._context = context
storm.logInfo("Spout instance starting...")
# Process the next tuple
def nextTuple(self):
# check if it reach at the EOF to close it
line = f.readline()
while line != "":
# Emit a random sentence
storm.logInfo("Emiting %s" % line)
storm.emit([line])
# Start the spout when it's invoked
SimSpout().run()
Another thing i want to ask about , i have shell script that take my data text file to make process on it to get the result , now if i want to make it in storm i should make spout open my data text file as its my data source and read line by line to take every line it got it to bolt that make a process on it which will contain the code in shell written in python , Right ?
Re: open file in spout with python
Posted by sam mohel <sa...@gmail.com>.
Is there any help ?
On Saturday, January 21, 2017, sam mohel <sa...@gmail.com> wrote:
> I'm new to python , i'm trying to make spout read from file line by line .
> i tried to write this statments but didn't work
>
>
> import storm
> import random
> # Define some sentences
>
> class SimSpout(storm.Spout):
> # Not much to do here for such a basic spout
> def initialize(self, conf, context):
> ## Open the file with read only permit
> f = open('mydata.txt', 'r')
> self._conf = conf
> self._context = context
>
> storm.logInfo("Spout instance starting...")
>
> # Process the next tuple
> def nextTuple(self):
> # check if it reach at the EOF to close it
> line = f.readline()
> while line != "":
> # Emit a random sentence
> storm.logInfo("Emiting %s" % line)
> storm.emit([line])
>
> # Start the spout when it's invoked
> SimSpout().run()
>
> Another thing i want to ask about , i have shell script that take my data
> text file to make process on it to get the result , now if i want to make
> it in storm i should make spout open my data text file as its my data
> source and read line by line to take every line it got it to bolt that make
> a process on it which will contain the code in shell written in python ,
> Right ?
>
Re: open file in spout with python
Posted by sam mohel <sa...@gmail.com>.
Is there any help ?
On Saturday, January 21, 2017, sam mohel <sa...@gmail.com> wrote:
> I'm new to python , i'm trying to make spout read from file line by line .
> i tried to write this statments but didn't work
>
>
> import storm
> import random
> # Define some sentences
>
> class SimSpout(storm.Spout):
> # Not much to do here for such a basic spout
> def initialize(self, conf, context):
> ## Open the file with read only permit
> f = open('mydata.txt', 'r')
> self._conf = conf
> self._context = context
>
> storm.logInfo("Spout instance starting...")
>
> # Process the next tuple
> def nextTuple(self):
> # check if it reach at the EOF to close it
> line = f.readline()
> while line != "":
> # Emit a random sentence
> storm.logInfo("Emiting %s" % line)
> storm.emit([line])
>
> # Start the spout when it's invoked
> SimSpout().run()
>
> Another thing i want to ask about , i have shell script that take my data
> text file to make process on it to get the result , now if i want to make
> it in storm i should make spout open my data text file as its my data
> source and read line by line to take every line it got it to bolt that make
> a process on it which will contain the code in shell written in python ,
> Right ?
>