You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Richard Huber <ri...@gmail.com> on 2015/05/11 08:23:33 UTC
Online docs for newbies or build/environment for storm-starter/WordCountTopology
broken?
Hey all -
Worked through all the online tutorial, etc, setup exactly as per
instructions. Everything seems to be working but the WordCountTopology
example with the python bolt. After some digging I can either copy:
storm-multilang/python/target/classes/resources/storm.py
to
examples/storm-starter/target/classes/resources/
Or of course set
PYTHONPATH=(path)/storm-multilang/python/target/classes/resources
And it appears as if it is running properly, though the subprocess
eventually dies without any output. (for more investigation on my part)
If storm is cloned clean, and all instructions run through WordCountToplogy
fails with:
Traceback (most recent call last):
File "splitsentence.py", line 16, in <module>
import storm
ImportError: No module named storm
Just want to bring this to the team's attention. Was there something
missing from the build that should have copied storm.py to
target/classes/resources? I surfed the various build files and didn't see
any reference to PYTHONPATH. Or perhaps newbie user error somewhere?
Cheers and thank you
Rich
Re: Online docs for newbies or build/environment for
storm-starter/WordCountTopology broken?
Posted by Richard Huber <ri...@gmail.com>.
Yes the master as of a week ago I think.
Cheers and thanks again
On Tue, May 12, 2015 at 9:44 AM, 임정택 <ka...@gmail.com> wrote:
> First of all, I assume that you use current master branch of Storm.
> Multilang packaging is not introduced to any releases yet, so issue exists
> only on master branch.
>
> And I report this to issue and post PR to fix.
> http://issues.apache.org/jira/browse/STORM-813
>
> Thanks for reporting. :)
>
> Jungtaek Lim (HeartSaVioR)
>
>
> 2015-05-12 9:20 GMT+09:00 Richard Huber <ri...@gmail.com>:
>
>> 임정택 -
>>
>> Thank you very much for the quick reply.... I thought so. The native or
>> clojure code runs fine, this is strictly a packaging for either local
>> directory or preparation of the environment issue for Python to locate it's
>> dependencies. Also a bit confusing when working through the issue as there
>> are actually two storm.py scripts where are completely different in the
>> release, one that will work from the source release in resources, and the
>> other in the root bin directory which gives the cryptic (cannot be run from
>> within the source checkout)
>>
>> Thanks again....
>>
>> Should we post this in a bug report?
>>
>> Rich
>>
>> On Mon, May 11, 2015 at 10:49 PM, 임정택 <ka...@gmail.com> wrote:
>>
>>> Hi!
>>>
>>> Recently Storm packages multilang files into Maven artifacts, and
>>> Supervisor extracts fat jar's resources to resource directory.
>>>
>>> Here's supervisor's log.
>>>
>>> 4584 [Thread-3] INFO backtype.storm.daemon.supervisor - Extracting
>>> resources from jar at ./target/flux-examples-0.11.0-SNAPSHOT.jar to
>>> /var/folders/3x/tz52s3jn2c71ts70k46_jdm40000gn/T//e2ff0cc0-6e81-43c6-8661-94f1ae5c3d39/supervisor/stormdist/shell-topology-1-1431350566/resources
>>>
>>> But AFAIK mvn exec:java doesn't do this for us, so for now we have to
>>> run multilang toplogy with standard way.
>>> If I'm right, it may be better to remove topologies which use multilang
>>> from storm-starter's README.markdown, "Running topologies with Maven".
>>>
>>> Hope this helps.
>>>
>>> Thanks.
>>> Jungtaek Lim (HeartSaVioR)
>>>
>>>
>>> 2015-05-11 15:23 GMT+09:00 Richard Huber <ri...@gmail.com>:
>>>
>>>> Hey all -
>>>>
>>>> Worked through all the online tutorial, etc, setup exactly as per
>>>> instructions. Everything seems to be working but the WordCountTopology
>>>> example with the python bolt. After some digging I can either copy:
>>>>
>>>> storm-multilang/python/target/classes/resources/storm.py
>>>> to
>>>> examples/storm-starter/target/classes/resources/
>>>>
>>>> Or of course set
>>>> PYTHONPATH=(path)/storm-multilang/python/target/classes/resources
>>>>
>>>> And it appears as if it is running properly, though the subprocess
>>>> eventually dies without any output. (for more investigation on my part)
>>>>
>>>> If storm is cloned clean, and all instructions run through
>>>> WordCountToplogy fails with:
>>>>
>>>> Traceback (most recent call last):
>>>> File "splitsentence.py", line 16, in <module>
>>>> import storm
>>>> ImportError: No module named storm
>>>>
>>>> Just want to bring this to the team's attention. Was there something
>>>> missing from the build that should have copied storm.py to
>>>> target/classes/resources? I surfed the various build files and didn't see
>>>> any reference to PYTHONPATH. Or perhaps newbie user error somewhere?
>>>>
>>>> Cheers and thank you
>>>>
>>>> Rich
>>>>
>>>
>>>
>>>
>>> --
>>> Name : 임 정택
>>> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
>>> Twitter : http://twitter.com/heartsavior
>>> LinkedIn : http://www.linkedin.com/in/heartsavior
>>>
>>
>>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>
Re: Online docs for newbies or build/environment for
storm-starter/WordCountTopology broken?
Posted by 임정택 <ka...@gmail.com>.
First of all, I assume that you use current master branch of Storm.
Multilang packaging is not introduced to any releases yet, so issue exists
only on master branch.
And I report this to issue and post PR to fix.
http://issues.apache.org/jira/browse/STORM-813
Thanks for reporting. :)
Jungtaek Lim (HeartSaVioR)
2015-05-12 9:20 GMT+09:00 Richard Huber <ri...@gmail.com>:
> 임정택 -
>
> Thank you very much for the quick reply.... I thought so. The native or
> clojure code runs fine, this is strictly a packaging for either local
> directory or preparation of the environment issue for Python to locate it's
> dependencies. Also a bit confusing when working through the issue as there
> are actually two storm.py scripts where are completely different in the
> release, one that will work from the source release in resources, and the
> other in the root bin directory which gives the cryptic (cannot be run from
> within the source checkout)
>
> Thanks again....
>
> Should we post this in a bug report?
>
> Rich
>
> On Mon, May 11, 2015 at 10:49 PM, 임정택 <ka...@gmail.com> wrote:
>
>> Hi!
>>
>> Recently Storm packages multilang files into Maven artifacts, and
>> Supervisor extracts fat jar's resources to resource directory.
>>
>> Here's supervisor's log.
>>
>> 4584 [Thread-3] INFO backtype.storm.daemon.supervisor - Extracting
>> resources from jar at ./target/flux-examples-0.11.0-SNAPSHOT.jar to
>> /var/folders/3x/tz52s3jn2c71ts70k46_jdm40000gn/T//e2ff0cc0-6e81-43c6-8661-94f1ae5c3d39/supervisor/stormdist/shell-topology-1-1431350566/resources
>>
>> But AFAIK mvn exec:java doesn't do this for us, so for now we have to run
>> multilang toplogy with standard way.
>> If I'm right, it may be better to remove topologies which use multilang
>> from storm-starter's README.markdown, "Running topologies with Maven".
>>
>> Hope this helps.
>>
>> Thanks.
>> Jungtaek Lim (HeartSaVioR)
>>
>>
>> 2015-05-11 15:23 GMT+09:00 Richard Huber <ri...@gmail.com>:
>>
>>> Hey all -
>>>
>>> Worked through all the online tutorial, etc, setup exactly as per
>>> instructions. Everything seems to be working but the WordCountTopology
>>> example with the python bolt. After some digging I can either copy:
>>>
>>> storm-multilang/python/target/classes/resources/storm.py
>>> to
>>> examples/storm-starter/target/classes/resources/
>>>
>>> Or of course set
>>> PYTHONPATH=(path)/storm-multilang/python/target/classes/resources
>>>
>>> And it appears as if it is running properly, though the subprocess
>>> eventually dies without any output. (for more investigation on my part)
>>>
>>> If storm is cloned clean, and all instructions run through
>>> WordCountToplogy fails with:
>>>
>>> Traceback (most recent call last):
>>> File "splitsentence.py", line 16, in <module>
>>> import storm
>>> ImportError: No module named storm
>>>
>>> Just want to bring this to the team's attention. Was there something
>>> missing from the build that should have copied storm.py to
>>> target/classes/resources? I surfed the various build files and didn't see
>>> any reference to PYTHONPATH. Or perhaps newbie user error somewhere?
>>>
>>> Cheers and thank you
>>>
>>> Rich
>>>
>>
>>
>>
>> --
>> Name : 임 정택
>> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
>> Twitter : http://twitter.com/heartsavior
>> LinkedIn : http://www.linkedin.com/in/heartsavior
>>
>
>
--
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior
Re: Online docs for newbies or build/environment for
storm-starter/WordCountTopology broken?
Posted by Richard Huber <ri...@gmail.com>.
임정택 -
Thank you very much for the quick reply.... I thought so. The native or
clojure code runs fine, this is strictly a packaging for either local
directory or preparation of the environment issue for Python to locate it's
dependencies. Also a bit confusing when working through the issue as there
are actually two storm.py scripts where are completely different in the
release, one that will work from the source release in resources, and the
other in the root bin directory which gives the cryptic (cannot be run from
within the source checkout)
Thanks again....
Should we post this in a bug report?
Rich
On Mon, May 11, 2015 at 10:49 PM, 임정택 <ka...@gmail.com> wrote:
> Hi!
>
> Recently Storm packages multilang files into Maven artifacts, and
> Supervisor extracts fat jar's resources to resource directory.
>
> Here's supervisor's log.
>
> 4584 [Thread-3] INFO backtype.storm.daemon.supervisor - Extracting
> resources from jar at ./target/flux-examples-0.11.0-SNAPSHOT.jar to
> /var/folders/3x/tz52s3jn2c71ts70k46_jdm40000gn/T//e2ff0cc0-6e81-43c6-8661-94f1ae5c3d39/supervisor/stormdist/shell-topology-1-1431350566/resources
>
> But AFAIK mvn exec:java doesn't do this for us, so for now we have to run
> multilang toplogy with standard way.
> If I'm right, it may be better to remove topologies which use multilang
> from storm-starter's README.markdown, "Running topologies with Maven".
>
> Hope this helps.
>
> Thanks.
> Jungtaek Lim (HeartSaVioR)
>
>
> 2015-05-11 15:23 GMT+09:00 Richard Huber <ri...@gmail.com>:
>
>> Hey all -
>>
>> Worked through all the online tutorial, etc, setup exactly as per
>> instructions. Everything seems to be working but the WordCountTopology
>> example with the python bolt. After some digging I can either copy:
>>
>> storm-multilang/python/target/classes/resources/storm.py
>> to
>> examples/storm-starter/target/classes/resources/
>>
>> Or of course set
>> PYTHONPATH=(path)/storm-multilang/python/target/classes/resources
>>
>> And it appears as if it is running properly, though the subprocess
>> eventually dies without any output. (for more investigation on my part)
>>
>> If storm is cloned clean, and all instructions run through
>> WordCountToplogy fails with:
>>
>> Traceback (most recent call last):
>> File "splitsentence.py", line 16, in <module>
>> import storm
>> ImportError: No module named storm
>>
>> Just want to bring this to the team's attention. Was there something
>> missing from the build that should have copied storm.py to
>> target/classes/resources? I surfed the various build files and didn't see
>> any reference to PYTHONPATH. Or perhaps newbie user error somewhere?
>>
>> Cheers and thank you
>>
>> Rich
>>
>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
>
Re: Online docs for newbies or build/environment for
storm-starter/WordCountTopology broken?
Posted by 임정택 <ka...@gmail.com>.
Hi!
Recently Storm packages multilang files into Maven artifacts, and
Supervisor extracts fat jar's resources to resource directory.
Here's supervisor's log.
4584 [Thread-3] INFO backtype.storm.daemon.supervisor - Extracting
resources from jar at ./target/flux-examples-0.11.0-SNAPSHOT.jar to
/var/folders/3x/tz52s3jn2c71ts70k46_jdm40000gn/T//e2ff0cc0-6e81-43c6-8661-94f1ae5c3d39/supervisor/stormdist/shell-topology-1-1431350566/resources
But AFAIK mvn exec:java doesn't do this for us, so for now we have to run
multilang toplogy with standard way.
If I'm right, it may be better to remove topologies which use multilang
from storm-starter's README.markdown, "Running topologies with Maven".
Hope this helps.
Thanks.
Jungtaek Lim (HeartSaVioR)
2015-05-11 15:23 GMT+09:00 Richard Huber <ri...@gmail.com>:
> Hey all -
>
> Worked through all the online tutorial, etc, setup exactly as per
> instructions. Everything seems to be working but the WordCountTopology
> example with the python bolt. After some digging I can either copy:
>
> storm-multilang/python/target/classes/resources/storm.py
> to
> examples/storm-starter/target/classes/resources/
>
> Or of course set
> PYTHONPATH=(path)/storm-multilang/python/target/classes/resources
>
> And it appears as if it is running properly, though the subprocess
> eventually dies without any output. (for more investigation on my part)
>
> If storm is cloned clean, and all instructions run through
> WordCountToplogy fails with:
>
> Traceback (most recent call last):
> File "splitsentence.py", line 16, in <module>
> import storm
> ImportError: No module named storm
>
> Just want to bring this to the team's attention. Was there something
> missing from the build that should have copied storm.py to
> target/classes/resources? I surfed the various build files and didn't see
> any reference to PYTHONPATH. Or perhaps newbie user error somewhere?
>
> Cheers and thank you
>
> Rich
>
--
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior