Re: [PR] [SPARK-55726][PYTHON][TEST] Add ASV microbenchmark for grouped map pandas UDF [spark]

["fangchenli (via GitHub)" <gi...@apache.org>]

fangchenli commented on code in PR #54533:
URL: https://github.com/apache/spark/pull/54533#discussion_r2867069161


##########
python/benchmarks/__init__.py:
##########
@@ -14,3 +14,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
+
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))

Review Comment:
   I mean installing the local source code in the current env. 
   
   First, install the current source code in editable mode via `pip install -e python/packaging/classic`
   If you do `pip list` or `conda list`, you should see something like
   `pyspark      4.2.0.dev0   ...../spark/python/packaging/classic`
   
   Then, when you set `--python=same`, `pyspark` is discoverable to asv.
   
   Without `--python=same`, `asv` would create a new venv, build a pyspark wheel of the current source code, then install the wheel in the new venv and run the benchmarks. The build and install commands are here:
   
   https://github.com/apache/spark/blob/168e5cf57147eed3e789b6d609a60396750094e4/python/benchmarks/asv.conf.json#L14-L19



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org