You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Russell Jurney <ru...@gmail.com> on 2011/12/15 02:23:16 UTC
Python for Avro doesn't build on OS X 10.6.8. Stuck.
I am unable to build Avro for Python on OS X 10.6.8 because python-snappy
fails to build. Is there a way around this?
I wrote a post here
http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-rubyabout
how to use Avro with Pig and Ruby. The code is here:
https://github.com/rjurney/Booting-the-Analytics-Application I am trying
to create Python instructions that parallel the Ruby ones.
I've tried multiple install methods, and I am unable to build
snappy-python, so the avro build fails. This is the error message:
Installed
/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg
Processing dependencies for avro==-AVRO-VERSION-
Searching for python-snappy
Reading http://pypi.python.org/simple/python-snappy/
Reading http://github.com/andrix/python-snappy
Best match: python-snappy 0.3.2
Downloading
http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
Processing python-snappy-0.3.2.tar.gz
Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
C/ObjC but not for C++
snappymodule.cc:31:22: error: snappy-c.h: No such file or directory
snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*,
PyObject*)’:
snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope
snappymodule.cc:62: error: expected `;' before ‘status’
snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not declared
in this scope
snappymodule.cc:79: error: ‘status’ was not declared in this scope
snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this scope
snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope
snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*,
PyObject*)’:
snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope
snappymodule.cc:107: error: expected `;' before ‘status’
snappymodule.cc:120: error: ‘status’ was not declared in this scope
snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared
in this scope
snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope
snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this
scope
snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope
snappymodule.cc: In function ‘PyObject*
snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’:
snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope
snappymodule.cc:151: error: expected `;' before ‘status’
snappymodule.cc:156: error: ‘status’ was not declared in this scope
snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not
declared in this scope
snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope
snappymodule.cc: At global scope:
snappymodule.cc:41: warning: ‘_state’ defined but not used
error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed
with exit status 1
--
Russell Jurney
twitter.com/rjurney
russell.jurney@gmail.com
datasyndrome.com
Re: Python for Avro doesn't build on OS X 10.6.8. Stuck.
Posted by Ken Krugler <kk...@transpac.com>.
On Dec 23, 2011, at 4:55pm, Russell Jurney wrote:
> Avro 1.53 doesn't have this issue? Does it use python-snappy?
No - this was (I think) right before Tom committed support for Snappy in Python.
-- Ken
>
> Russell Jurney
> twitter.com/rjurney
> russell.jurney@gmail.com
> datasyndrome.com
>
> On Dec 23, 2011, at 7:05 PM, Ken Krugler <kk...@transpac.com> wrote:
>
>> I installed brew, then ran 'brew install snappy', which worked.
>>
>> But 'sudo python setup.py install' still fails, though with a different problem (I was getting the same compilation errors as below, now I get a broken pipe error).
>>
>>> running install
>>> running bdist_egg
>>> running egg_info
>>> writing requirements to avro.egg-info/requires.txt
>>> writing avro.egg-info/PKG-INFO
>>> writing top-level names to avro.egg-info/top_level.txt
>>> writing dependency_links to avro.egg-info/dependency_links.txt
>>> reading manifest file 'avro.egg-info/SOURCES.txt'
>>> writing manifest file 'avro.egg-info/SOURCES.txt'
>>> installing library code to build/bdist.macosx-10.6-universal/egg
>>> running install_lib
>>> running build_py
>>> creating build/bdist.macosx-10.6-universal/egg
>>> creating build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/__init__.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/datafile.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/io.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/ipc.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/protocol.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/schema.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/tool.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> copying build/lib/avro/txipc.py -> build/bdist.macosx-10.6-universal/egg/avro
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/__init__.py to __init__.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/datafile.py to datafile.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/io.py to io.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/ipc.py to ipc.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/protocol.py to protocol.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/schema.py to schema.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/tool.py to tool.pyc
>>> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/txipc.py to txipc.pyc
>>> creating build/bdist.macosx-10.6-universal/egg/EGG-INFO
>>> installing scripts to build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
>>> running install_scripts
>>> running build_scripts
>>> creating build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
>>> copying build/scripts-2.6/avro -> build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
>>> changing mode of build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts/avro to 755
>>> copying avro.egg-info/PKG-INFO -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
>>> copying avro.egg-info/SOURCES.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
>>> copying avro.egg-info/dependency_links.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
>>> copying avro.egg-info/requires.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
>>> copying avro.egg-info/top_level.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
>>> zip_safe flag not set; analyzing archive contents...
>>> creating 'dist/avro-1.6.1-py2.6.egg' and adding 'build/bdist.macosx-10.6-universal/egg' to it
>>> removing 'build/bdist.macosx-10.6-universal/egg' (and everything under it)
>>> Processing avro-1.6.1-py2.6.egg
>>> Removing /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg
>>> Copying avro-1.6.1-py2.6.egg to /Library/Python/2.6/site-packages
>>> avro 1.6.1 is already the active version in easy-install.pth
>>> Installing avro script to /usr/local/bin
>>>
>>> Installed /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg
>>> Processing dependencies for avro==1.6.1
>>> Searching for python-snappy
>>> Reading http://pypi.python.org/simple/python-snappy/
>>> Reading http://github.com/andrix/python-snappy
>>> Best match: python-snappy 0.3.2
>>> Downloading http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
>>> Processing python-snappy-0.3.2.tar.gz
>>> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-jARda8/python-snappy-0.3.2/egg-dist-tmp-DDcMTW
>>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
>>> snappymodule.cc:41: warning: ‘_state’ defined but not used
>>> /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
>>> Installed assemblers are:
>>> /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
>>> /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
>>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
>>> snappymodule.cc:41: warning: ‘_state’ defined but not used
>>> snappymodule.cc:246: fatal error: error writing to -: Broken pipe
>>> compilation terminated.
>>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
>>> snappymodule.cc:41: warning: ‘_state’ defined but not used
>>> lipo: can't open input file: /var/tmp//cc1RAU1Q.out (No such file or directory)
>>> error: Setup script exited with error: command 'gcc-4.2' failed with exit status 1
>>
>>
>> Note that I have to run as sudo, otherwise I get:
>>
>> Installing avro script to /usr/local/bin
>> error: /usr/local/bin/avro: Permission denied
>>
>> I've added my vote to https://issues.apache.org/jira/browse/AVRO-981, and a comment about why I think it's good to fix this issue.
>>
>> In the meantime, we're going to use Avro 1.5.3
>>
>> Thanks,
>>
>> -- Ken
>>
>> On Dec 14, 2011, at 7:07pm, Russell Jurney wrote:
>>
>>> I installed the snappy macport, and I still can't get snappy-python to build. I'll try switching to brew.
>>>
>>> On Wed, Dec 14, 2011 at 5:46 PM, Joe Crobak <jo...@gmail.com> wrote:
>>> It appears that the python-snappy library assumes that the snappy library has already been installed. I saw similar errors, but once I installed snappy (I use homebrew, so 'brew install snappy') python-snappy installs fine.
>>>
>>> HTH,
>>> Joe
>>>
>>>
>>> On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney <ru...@gmail.com> wrote:
>>> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy fails to build. Is there a way around this?
>>>
>>> I wrote a post here http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-ruby about how to use Avro with Pig and Ruby. The code is here: https://github.com/rjurney/Booting-the-Analytics-Application I am trying to create Python instructions that parallel the Ruby ones.
>>>
>>> I've tried multiple install methods, and I am unable to build snappy-python, so the avro build fails. This is the error message:
>>>
>>> Installed /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg
>>> Processing dependencies for avro==-AVRO-VERSION-
>>> Searching for python-snappy
>>> Reading http://pypi.python.org/simple/python-snappy/
>>> Reading http://github.com/andrix/python-snappy
>>> Best match: python-snappy 0.3.2
>>> Downloading http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
>>> Processing python-snappy-0.3.2.tar.gz
>>> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j
>>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
>>> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory
>>> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*, PyObject*)’:
>>> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope
>>> snappymodule.cc:62: error: expected `;' before ‘status’
>>> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not declared in this scope
>>> snappymodule.cc:79: error: ‘status’ was not declared in this scope
>>> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this scope
>>> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope
>>> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*, PyObject*)’:
>>> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope
>>> snappymodule.cc:107: error: expected `;' before ‘status’
>>> snappymodule.cc:120: error: ‘status’ was not declared in this scope
>>> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared in this scope
>>> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope
>>> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this scope
>>> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope
>>> snappymodule.cc: In function ‘PyObject* snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’:
>>> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope
>>> snappymodule.cc:151: error: expected `;' before ‘status’
>>> snappymodule.cc:156: error: ‘status’ was not declared in this scope
>>> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not declared in this scope
>>> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope
>>> snappymodule.cc: At global scope:
>>> snappymodule.cc:41: warning: ‘_state’ defined but not used
>>> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed with exit status 1
>>> --
>>
>> --------------------------
>> Ken Krugler
>> http://www.scaleunlimited.com
>> custom big data solutions & training
>> Hadoop, Cascading, Mahout & Solr
>>
>>
>>
>>
--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Re: Python for Avro doesn't build on OS X 10.6.8. Stuck.
Posted by Russell Jurney <ru...@gmail.com>.
Avro 1.53 doesn't have this issue? Does it use python-snappy?
Russell Jurney
twitter.com/rjurney
russell.jurney@gmail.com
datasyndrome.com
On Dec 23, 2011, at 7:05 PM, Ken Krugler <kk...@transpac.com>
wrote:
I installed brew, then ran 'brew install snappy', which worked.
But 'sudo python setup.py install' still fails, though with a different
problem (I was getting the same compilation errors as below, now I get a
broken pipe error).
running install
running bdist_egg
running egg_info
writing requirements to avro.egg-info/requires.txt
writing avro.egg-info/PKG-INFO
writing top-level names to avro.egg-info/top_level.txt
writing dependency_links to avro.egg-info/dependency_links.txt
reading manifest file 'avro.egg-info/SOURCES.txt'
writing manifest file 'avro.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-universal/egg
running install_lib
running build_py
creating build/bdist.macosx-10.6-universal/egg
creating build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/__init__.py ->
build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/datafile.py ->
build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/io.py -> build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/ipc.py -> build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/protocol.py ->
build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/schema.py ->
build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/tool.py -> build/bdist.macosx-10.6-universal/egg/avro
copying build/lib/avro/txipc.py ->
build/bdist.macosx-10.6-universal/egg/avro
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/__init__.py to
__init__.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/datafile.py to
datafile.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/io.py to io.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/ipc.py to ipc.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/protocol.py to
protocol.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/schema.py to
schema.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/tool.py to
tool.pyc
byte-compiling build/bdist.macosx-10.6-universal/egg/avro/txipc.py to
txipc.pyc
creating build/bdist.macosx-10.6-universal/egg/EGG-INFO
installing scripts to build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
running install_scripts
running build_scripts
creating build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
copying build/scripts-2.6/avro ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
changing mode of
build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts/avro to 755
copying avro.egg-info/PKG-INFO ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying avro.egg-info/SOURCES.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying avro.egg-info/dependency_links.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying avro.egg-info/requires.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
copying avro.egg-info/top_level.txt ->
build/bdist.macosx-10.6-universal/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/avro-1.6.1-py2.6.egg' and adding
'build/bdist.macosx-10.6-universal/egg' to it
removing 'build/bdist.macosx-10.6-universal/egg' (and everything under it)
Processing avro-1.6.1-py2.6.egg
Removing /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg
Copying avro-1.6.1-py2.6.egg to /Library/Python/2.6/site-packages
avro 1.6.1 is already the active version in easy-install.pth
Installing avro script to /usr/local/bin
Installed /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg
Processing dependencies for avro==1.6.1
Searching for python-snappy
Reading http://pypi.python.org/simple/python-snappy/
Reading http://github.com/andrix/python-snappy
Best match: python-snappy 0.3.2
Downloading
http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
Processing python-snappy-0.3.2.tar.gz
Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-jARda8/python-snappy-0.3.2/egg-dist-tmp-DDcMTW
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
C/ObjC but not for C++
snappymodule.cc:41: warning: ‘_state’ defined but not used
/usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler
(/usr/bin/../libexec/gcc/darwin/ppc/as or
/usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not
installed
Installed assemblers are:
/usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
/usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
C/ObjC but not for C++
snappymodule.cc:41: warning: ‘_state’ defined but not used
snappymodule.cc:246: fatal error: error writing to -: Broken pipe
compilation terminated.
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
C/ObjC but not for C++
snappymodule.cc:41: warning: ‘_state’ defined but not used
lipo: can't open input file: /var/tmp//cc1RAU1Q.out (No such file or
directory)
error: Setup script exited with error: command 'gcc-4.2' failed with exit
status 1
Note that I have to run as sudo, otherwise I get:
Installing avro script to /usr/local/bin
error: /usr/local/bin/avro: Permission denied
I've added my vote to https://issues.apache.org/jira/browse/AVRO-981, and a
comment about why I think it's good to fix this issue.
In the meantime, we're going to use Avro 1.5.3
Thanks,
-- Ken
On Dec 14, 2011, at 7:07pm, Russell Jurney wrote:
I installed the snappy macport, and I still can't get snappy-python to
build. I'll try switching to brew.
On Wed, Dec 14, 2011 at 5:46 PM, Joe Crobak <jo...@gmail.com> wrote:
> It appears that the python-snappy library assumes that the snappy library
> has already been installed. I saw similar errors, but once I installed
> snappy (I use homebrew, so 'brew install snappy') python-snappy installs
> fine.
>
> HTH,
> Joe
>
>
> On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney <ru...@gmail.com>wrote:
>
>> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy
>> fails to build. Is there a way around this?
>>
>> I wrote a post here
>> http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-rubyabout how to use Avro with Pig and Ruby. The code is here:
>> https://github.com/rjurney/Booting-the-Analytics-Application I am
>> trying to create Python instructions that parallel the Ruby ones.
>>
>> I've tried multiple install methods, and I am unable to build
>> snappy-python, so the avro build fails. This is the error message:
>>
>> Installed
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg
>> Processing dependencies for avro==-AVRO-VERSION-
>> Searching for python-snappy
>> Reading http://pypi.python.org/simple/python-snappy/
>> Reading http://github.com/andrix/python-snappy
>> Best match: python-snappy 0.3.2
>> Downloading
>> http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
>> Processing python-snappy-0.3.2.tar.gz
>> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir
>> /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j
>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
>> C/ObjC but not for C++
>> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory
>> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*,
>> PyObject*)’:
>> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope
>> snappymodule.cc:62: error: expected `;' before ‘status’
>> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not
>> declared in this scope
>> snappymodule.cc:79: error: ‘status’ was not declared in this scope
>> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this
>> scope
>> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*,
>> PyObject*)’:
>> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope
>> snappymodule.cc:107: error: expected `;' before ‘status’
>> snappymodule.cc:120: error: ‘status’ was not declared in this scope
>> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared
>> in this scope
>> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this
>> scope
>> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc: In function ‘PyObject*
>> snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’:
>> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope
>> snappymodule.cc:151: error: expected `;' before ‘status’
>> snappymodule.cc:156: error: ‘status’ was not declared in this scope
>> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not
>> declared in this scope
>> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc: At global scope:
>> snappymodule.cc:41: warning: ‘_state’ defined but not used
>> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed
>> with exit status 1
>>
>> --
>>
>
--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Re: Python for Avro doesn't build on OS X 10.6.8. Stuck.
Posted by Ken Krugler <kk...@transpac.com>.
I installed brew, then ran 'brew install snappy', which worked.
But 'sudo python setup.py install' still fails, though with a different problem (I was getting the same compilation errors as below, now I get a broken pipe error).
> running install
> running bdist_egg
> running egg_info
> writing requirements to avro.egg-info/requires.txt
> writing avro.egg-info/PKG-INFO
> writing top-level names to avro.egg-info/top_level.txt
> writing dependency_links to avro.egg-info/dependency_links.txt
> reading manifest file 'avro.egg-info/SOURCES.txt'
> writing manifest file 'avro.egg-info/SOURCES.txt'
> installing library code to build/bdist.macosx-10.6-universal/egg
> running install_lib
> running build_py
> creating build/bdist.macosx-10.6-universal/egg
> creating build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/__init__.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/datafile.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/io.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/ipc.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/protocol.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/schema.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/tool.py -> build/bdist.macosx-10.6-universal/egg/avro
> copying build/lib/avro/txipc.py -> build/bdist.macosx-10.6-universal/egg/avro
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/__init__.py to __init__.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/datafile.py to datafile.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/io.py to io.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/ipc.py to ipc.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/protocol.py to protocol.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/schema.py to schema.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/tool.py to tool.pyc
> byte-compiling build/bdist.macosx-10.6-universal/egg/avro/txipc.py to txipc.pyc
> creating build/bdist.macosx-10.6-universal/egg/EGG-INFO
> installing scripts to build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
> running install_scripts
> running build_scripts
> creating build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
> copying build/scripts-2.6/avro -> build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts
> changing mode of build/bdist.macosx-10.6-universal/egg/EGG-INFO/scripts/avro to 755
> copying avro.egg-info/PKG-INFO -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
> copying avro.egg-info/SOURCES.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
> copying avro.egg-info/dependency_links.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
> copying avro.egg-info/requires.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
> copying avro.egg-info/top_level.txt -> build/bdist.macosx-10.6-universal/egg/EGG-INFO
> zip_safe flag not set; analyzing archive contents...
> creating 'dist/avro-1.6.1-py2.6.egg' and adding 'build/bdist.macosx-10.6-universal/egg' to it
> removing 'build/bdist.macosx-10.6-universal/egg' (and everything under it)
> Processing avro-1.6.1-py2.6.egg
> Removing /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg
> Copying avro-1.6.1-py2.6.egg to /Library/Python/2.6/site-packages
> avro 1.6.1 is already the active version in easy-install.pth
> Installing avro script to /usr/local/bin
>
> Installed /Library/Python/2.6/site-packages/avro-1.6.1-py2.6.egg
> Processing dependencies for avro==1.6.1
> Searching for python-snappy
> Reading http://pypi.python.org/simple/python-snappy/
> Reading http://github.com/andrix/python-snappy
> Best match: python-snappy 0.3.2
> Downloading http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
> Processing python-snappy-0.3.2.tar.gz
> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-jARda8/python-snappy-0.3.2/egg-dist-tmp-DDcMTW
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
> snappymodule.cc:41: warning: ‘_state’ defined but not used
> /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
> Installed assemblers are:
> /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
> /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
> snappymodule.cc:41: warning: ‘_state’ defined but not used
> snappymodule.cc:246: fatal error: error writing to -: Broken pipe
> compilation terminated.
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
> snappymodule.cc:41: warning: ‘_state’ defined but not used
> lipo: can't open input file: /var/tmp//cc1RAU1Q.out (No such file or directory)
> error: Setup script exited with error: command 'gcc-4.2' failed with exit status 1
Note that I have to run as sudo, otherwise I get:
Installing avro script to /usr/local/bin
error: /usr/local/bin/avro: Permission denied
I've added my vote to https://issues.apache.org/jira/browse/AVRO-981, and a comment about why I think it's good to fix this issue.
In the meantime, we're going to use Avro 1.5.3
Thanks,
-- Ken
On Dec 14, 2011, at 7:07pm, Russell Jurney wrote:
> I installed the snappy macport, and I still can't get snappy-python to build. I'll try switching to brew.
>
> On Wed, Dec 14, 2011 at 5:46 PM, Joe Crobak <jo...@gmail.com> wrote:
> It appears that the python-snappy library assumes that the snappy library has already been installed. I saw similar errors, but once I installed snappy (I use homebrew, so 'brew install snappy') python-snappy installs fine.
>
> HTH,
> Joe
>
>
> On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney <ru...@gmail.com> wrote:
> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy fails to build. Is there a way around this?
>
> I wrote a post here http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-ruby about how to use Avro with Pig and Ruby. The code is here: https://github.com/rjurney/Booting-the-Analytics-Application I am trying to create Python instructions that parallel the Ruby ones.
>
> I've tried multiple install methods, and I am unable to build snappy-python, so the avro build fails. This is the error message:
>
> Installed /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg
> Processing dependencies for avro==-AVRO-VERSION-
> Searching for python-snappy
> Reading http://pypi.python.org/simple/python-snappy/
> Reading http://github.com/andrix/python-snappy
> Best match: python-snappy 0.3.2
> Downloading http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
> Processing python-snappy-0.3.2.tar.gz
> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for C/ObjC but not for C++
> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory
> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*, PyObject*)’:
> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope
> snappymodule.cc:62: error: expected `;' before ‘status’
> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not declared in this scope
> snappymodule.cc:79: error: ‘status’ was not declared in this scope
> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this scope
> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*, PyObject*)’:
> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope
> snappymodule.cc:107: error: expected `;' before ‘status’
> snappymodule.cc:120: error: ‘status’ was not declared in this scope
> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared in this scope
> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this scope
> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc: In function ‘PyObject* snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’:
> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope
> snappymodule.cc:151: error: expected `;' before ‘status’
> snappymodule.cc:156: error: ‘status’ was not declared in this scope
> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not declared in this scope
> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc: At global scope:
> snappymodule.cc:41: warning: ‘_state’ defined but not used
> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed with exit status 1
> --
--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Re: Python for Avro doesn't build on OS X 10.6.8. Stuck.
Posted by Russell Jurney <ru...@gmail.com>.
I installed the snappy macport, and I still can't get snappy-python to
build. I'll try switching to brew.
On Wed, Dec 14, 2011 at 5:46 PM, Joe Crobak <jo...@gmail.com> wrote:
> It appears that the python-snappy library assumes that the snappy library
> has already been installed. I saw similar errors, but once I installed
> snappy (I use homebrew, so 'brew install snappy') python-snappy installs
> fine.
>
> HTH,
> Joe
>
>
> On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney <ru...@gmail.com>wrote:
>
>> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy
>> fails to build. Is there a way around this?
>>
>> I wrote a post here
>> http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-rubyabout how to use Avro with Pig and Ruby. The code is here:
>> https://github.com/rjurney/Booting-the-Analytics-Application I am
>> trying to create Python instructions that parallel the Ruby ones.
>>
>> I've tried multiple install methods, and I am unable to build
>> snappy-python, so the avro build fails. This is the error message:
>>
>> Installed
>> /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg
>> Processing dependencies for avro==-AVRO-VERSION-
>> Searching for python-snappy
>> Reading http://pypi.python.org/simple/python-snappy/
>> Reading http://github.com/andrix/python-snappy
>> Best match: python-snappy 0.3.2
>> Downloading
>> http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
>> Processing python-snappy-0.3.2.tar.gz
>> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir
>> /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j
>> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
>> C/ObjC but not for C++
>> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory
>> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*,
>> PyObject*)’:
>> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope
>> snappymodule.cc:62: error: expected `;' before ‘status’
>> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not
>> declared in this scope
>> snappymodule.cc:79: error: ‘status’ was not declared in this scope
>> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this
>> scope
>> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*,
>> PyObject*)’:
>> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope
>> snappymodule.cc:107: error: expected `;' before ‘status’
>> snappymodule.cc:120: error: ‘status’ was not declared in this scope
>> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared
>> in this scope
>> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this
>> scope
>> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc: In function ‘PyObject*
>> snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’:
>> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope
>> snappymodule.cc:151: error: expected `;' before ‘status’
>> snappymodule.cc:156: error: ‘status’ was not declared in this scope
>> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not
>> declared in this scope
>> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope
>> snappymodule.cc: At global scope:
>> snappymodule.cc:41: warning: ‘_state’ defined but not used
>> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed
>> with exit status 1
>>
>> --
>> Russell Jurney
>> twitter.com/rjurney
>> russell.jurney@gmail.com
>> datasyndrome.com
>>
>
>
--
Russell Jurney
twitter.com/rjurney
russell.jurney@gmail.com
datasyndrome.com
Re: Python for Avro doesn't build on OS X 10.6.8. Stuck.
Posted by Joe Crobak <jo...@gmail.com>.
It appears that the python-snappy library assumes that the snappy library
has already been installed. I saw similar errors, but once I installed
snappy (I use homebrew, so 'brew install snappy') python-snappy installs
fine.
HTH,
Joe
On Wed, Dec 14, 2011 at 8:23 PM, Russell Jurney <ru...@gmail.com>wrote:
> I am unable to build Avro for Python on OS X 10.6.8 because python-snappy
> fails to build. Is there a way around this?
>
> I wrote a post here
> http://datasyndrome.com/post/13707537045/booting-the-analytics-application-events-rubyabout how to use Avro with Pig and Ruby. The code is here:
> https://github.com/rjurney/Booting-the-Analytics-Application I am trying
> to create Python instructions that parallel the Ruby ones.
>
> I've tried multiple install methods, and I am unable to build
> snappy-python, so the avro build fails. This is the error message:
>
> Installed
> /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/avro-_AVRO_VERSION_-py2.6.egg
> Processing dependencies for avro==-AVRO-VERSION-
> Searching for python-snappy
> Reading http://pypi.python.org/simple/python-snappy/
> Reading http://github.com/andrix/python-snappy
> Best match: python-snappy 0.3.2
> Downloading
> http://pypi.python.org/packages/source/p/python-snappy/python-snappy-0.3.2.tar.gz#md5=94ec3eb54a780fac3b15a6c141af973f
> Processing python-snappy-0.3.2.tar.gz
> Running python-snappy-0.3.2/setup.py -q bdist_egg --dist-dir
> /tmp/easy_install-0KEFAv/python-snappy-0.3.2/egg-dist-tmp-1BPj3j
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
> C/ObjC but not for C++
> snappymodule.cc:31:22: error: snappy-c.h: No such file or directory
> snappymodule.cc: In function ‘PyObject* snappy__compress(PyObject*,
> PyObject*)’:
> snappymodule.cc:62: error: ‘snappy_status’ was not declared in this scope
> snappymodule.cc:62: error: expected `;' before ‘status’
> snappymodule.cc:75: error: ‘snappy_max_compressed_length’ was not declared
> in this scope
> snappymodule.cc:79: error: ‘status’ was not declared in this scope
> snappymodule.cc:79: error: ‘snappy_compress’ was not declared in this scope
> snappymodule.cc:81: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc: In function ‘PyObject* snappy__uncompress(PyObject*,
> PyObject*)’:
> snappymodule.cc:107: error: ‘snappy_status’ was not declared in this scope
> snappymodule.cc:107: error: expected `;' before ‘status’
> snappymodule.cc:120: error: ‘status’ was not declared in this scope
> snappymodule.cc:120: error: ‘snappy_uncompressed_length’ was not declared
> in this scope
> snappymodule.cc:121: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc:128: error: ‘snappy_uncompress’ was not declared in this
> scope
> snappymodule.cc:129: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc: In function ‘PyObject*
> snappy__is_valid_compressed_buffer(PyObject*, PyObject*)’:
> snappymodule.cc:151: error: ‘snappy_status’ was not declared in this scope
> snappymodule.cc:151: error: expected `;' before ‘status’
> snappymodule.cc:156: error: ‘status’ was not declared in this scope
> snappymodule.cc:156: error: ‘snappy_validate_compressed_buffer’ was not
> declared in this scope
> snappymodule.cc:157: error: ‘SNAPPY_OK’ was not declared in this scope
> snappymodule.cc: At global scope:
> snappymodule.cc:41: warning: ‘_state’ defined but not used
> error: Setup script exited with error: command '/usr/bin/gcc-4.2' failed
> with exit status 1
>
> --
> Russell Jurney
> twitter.com/rjurney
> russell.jurney@gmail.com
> datasyndrome.com
>