You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Drew Moore <dr...@gmail.com> on 2020/08/30 19:13:12 UTC
Question on pyarrow compute
Hello,
My apologies if this isn't the right place to contact.
I’m interested in exploring the Python binding. I was wondering how I could
access the compute functions (other than sum) from pyarrow.
I see this Jira ticket: https://issues.apache.org/jira/browse/ARROW-7871,
and it references a pyarrow.compute module, which I don’t seem to have in
my version of pyarrow (version 0.15.1, installed from conda). Do I need to
install a nightly* build to get access to it?
Thanks so much! I'm looking forward to following this project.
Best,
Drew Moore
-----
*I've tried to install the nightly builds with no success, though it may be
due to my weak conda-fu. Here's what I've tried. (I also tried installing
python=3.8, but that also did not work.)
(pyarrow_env) Drews-MacBook-Pro:~ drewmoore$ conda list python
# packages in environment at /Library/miniconda3/envs/pyarrow_env:
#
# Name Version Build Channel
python 3.7.7 hf48f09d_4
(pyarrow_env) Drews-MacBook-Pro:~ drewmoore$ conda install -c
arrow-nightlies pyarrow=1.1.0.dev137=py37h1234567_0_cpu --verbose
Collecting package metadata (current_repodata.json): ...working... Unable
to retrieve repodata (response: 404) for
https://conda.anaconda.org/arrow-nightlies/osx-64/current_repodata.json
done
Solving environment: ...working... failed with initial frozen solve.
Retrying with flexible solve.
Solving environment: ...working... failed with repodata from
current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve.
Retrying with flexible solve.
Solving environment: ...working...
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
Traceback (most recent call last):
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/cli/install.py",
line 261, in install
unlink_link_transaction = solver.solve_for_transaction(
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
114, in solve_for_transaction
unlink_precs, link_precs = self.solve_for_diff(update_modifier,
deps_modifier,
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
157, in solve_for_diff
final_precs = self.solve_final_state(update_modifier, deps_modifier,
prune, ignore_pinned,
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
281, in solve_final_state
ssc = self._run_sat(ssc)
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/common/io.py", line
88, in decorated
return f(*args, **kwds)
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
815, in _run_sat
ssc.solution_precs = ssc.r.solve(tuple(final_environment_specs),
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/common/io.py", line
88, in decorated
return f(*args, **kwds)
File "/Library/miniconda3/lib/python3.8/site-packages/conda/resolve.py",
line 1319, in solve
raise UnsatisfiableError({})
conda.exceptions.UnsatisfiableError:
Did not find conflicting dependencies. If you would like to know which
packages conflict ensure that you have enabled unsatisfiable hints.
conda config --set unsatisfiable_hints True
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/exceptions.py", line
1079, in __call__
return func(*args, **kwargs)
File "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/main.py",
line 84, in _main
exit_code = do_call(args, p)
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/cli/conda_argparse.py",
line 83, in do_call
return getattr(module, func_name)(args, parser)
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/cli/main_install.py",
line 20, in execute
install(args, parser, 'install')
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/cli/install.py",
line 308, in install
raise e
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/cli/install.py",
line 295, in install
unlink_link_transaction = solver.solve_for_transaction(
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
114, in solve_for_transaction
unlink_precs, link_precs = self.solve_for_diff(update_modifier,
deps_modifier,
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
157, in solve_for_diff
final_precs = self.solve_final_state(update_modifier, deps_modifier,
prune, ignore_pinned,
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
275, in solve_final_state
ssc = self._add_specs(ssc)
File
"/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
704, in _add_specs
ssc.r.find_conflicts(spec_set)
File "/Library/miniconda3/lib/python3.8/site-packages/conda/resolve.py",
line 352, in find_conflicts
raise UnsatisfiableError(bad_deps, strict=strict_channel_priority)
conda.exceptions.UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your
environment:
Specifications:
- pyarrow==1.1.0.dev137=py37h1234567_0_cpu ->
python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0']
Your python: python=3.7
If python is on the left-most side of the chain, that's the version you've
asked for.
When python appears to the right, that indicates that the thing on the left
is somehow
not available for the python version you are constrained to. Note that
conda will not
change your python version to a different minor version unless you
explicitly specify
that.
Re: Question on pyarrow compute
Posted by Andrew Wieteska <an...@gmail.com>.
Hi Drew,
Pyarrow's compute module should be accessible in 0.17+ so updating your
installation should solve the problem. I don't think you need the nightly
build, just a newer release.
For updating, it might be you will need to use the conda-forge channel, as
it is possible that the default Anaconda channel does not have the more
recent versions of pyarrow available (`conda update pyarrow --channel
conda-forge`).
Best,
Andrew
(note: I am sending this to the user mailing list as well, as that seems
like the better place for this question)
On Sun, Aug 30, 2020 at 3:13 PM Drew Moore <dr...@gmail.com> wrote:
> Hello,
>
> My apologies if this isn't the right place to contact.
>
> I’m interested in exploring the Python binding. I was wondering how I could
> access the compute functions (other than sum) from pyarrow.
>
> I see this Jira ticket: https://issues.apache.org/jira/browse/ARROW-7871,
> and it references a pyarrow.compute module, which I don’t seem to have in
> my version of pyarrow (version 0.15.1, installed from conda). Do I need to
> install a nightly* build to get access to it?
>
> Thanks so much! I'm looking forward to following this project.
>
>
> Best,
> Drew Moore
>
>
> -----
>
> *I've tried to install the nightly builds with no success, though it may be
> due to my weak conda-fu. Here's what I've tried. (I also tried installing
> python=3.8, but that also did not work.)
>
> (pyarrow_env) Drews-MacBook-Pro:~ drewmoore$ conda list python
> # packages in environment at /Library/miniconda3/envs/pyarrow_env:
> #
> # Name Version Build Channel
> python 3.7.7 hf48f09d_4
>
> (pyarrow_env) Drews-MacBook-Pro:~ drewmoore$ conda install -c
> arrow-nightlies pyarrow=1.1.0.dev137=py37h1234567_0_cpu --verbose
> Collecting package metadata (current_repodata.json): ...working... Unable
> to retrieve repodata (response: 404) for
> https://conda.anaconda.org/arrow-nightlies/osx-64/current_repodata.json
>
> done
> Solving environment: ...working... failed with initial frozen solve.
> Retrying with flexible solve.
> Solving environment: ...working... failed with repodata from
> current_repodata.json, will retry with next repodata source.
> Collecting package metadata (repodata.json): ...working... done
> Solving environment: ...working... failed with initial frozen solve.
> Retrying with flexible solve.
> Solving environment: ...working...
> Found conflicts! Looking for incompatible packages.
> This can take several minutes. Press CTRL-C to abort.
> failed
>
> Traceback (most recent call last):
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/install.py",
> line 261, in install
> unlink_link_transaction = solver.solve_for_transaction(
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 114, in solve_for_transaction
> unlink_precs, link_precs = self.solve_for_diff(update_modifier,
> deps_modifier,
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 157, in solve_for_diff
> final_precs = self.solve_final_state(update_modifier, deps_modifier,
> prune, ignore_pinned,
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 281, in solve_final_state
> ssc = self._run_sat(ssc)
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/common/io.py", line
> 88, in decorated
> return f(*args, **kwds)
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 815, in _run_sat
> ssc.solution_precs = ssc.r.solve(tuple(final_environment_specs),
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/common/io.py", line
> 88, in decorated
> return f(*args, **kwds)
> File "/Library/miniconda3/lib/python3.8/site-packages/conda/resolve.py",
> line 1319, in solve
> raise UnsatisfiableError({})
> conda.exceptions.UnsatisfiableError:
> Did not find conflicting dependencies. If you would like to know which
> packages conflict ensure that you have enabled unsatisfiable hints.
>
> conda config --set unsatisfiable_hints True
>
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/exceptions.py", line
> 1079, in __call__
> return func(*args, **kwargs)
> File "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/main.py",
> line 84, in _main
> exit_code = do_call(args, p)
> File
>
> "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/conda_argparse.py",
> line 83, in do_call
> return getattr(module, func_name)(args, parser)
> File
>
> "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/main_install.py",
> line 20, in execute
> install(args, parser, 'install')
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/install.py",
> line 308, in install
> raise e
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/cli/install.py",
> line 295, in install
> unlink_link_transaction = solver.solve_for_transaction(
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 114, in solve_for_transaction
> unlink_precs, link_precs = self.solve_for_diff(update_modifier,
> deps_modifier,
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 157, in solve_for_diff
> final_precs = self.solve_final_state(update_modifier, deps_modifier,
> prune, ignore_pinned,
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 275, in solve_final_state
> ssc = self._add_specs(ssc)
> File
> "/Library/miniconda3/lib/python3.8/site-packages/conda/core/solve.py", line
> 704, in _add_specs
> ssc.r.find_conflicts(spec_set)
> File "/Library/miniconda3/lib/python3.8/site-packages/conda/resolve.py",
> line 352, in find_conflicts
> raise UnsatisfiableError(bad_deps, strict=strict_channel_priority)
> conda.exceptions.UnsatisfiableError: The following specifications were
> found
> to be incompatible with the existing python installation in your
> environment:
>
> Specifications:
>
> - pyarrow==1.1.0.dev137=py37h1234567_0_cpu ->
>
> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0']
>
> Your python: python=3.7
>
> If python is on the left-most side of the chain, that's the version you've
> asked for.
> When python appears to the right, that indicates that the thing on the left
> is somehow
> not available for the python version you are constrained to. Note that
> conda will not
> change your python version to a different minor version unless you
> explicitly specify
> that.
>