[GitHub] [arrow-datafusion] yjshen commented on pull request #2217: Add single line description of ExecutionPlan (#2216)

[GitBox <gi...@apache.org>]

yjshen commented on PR #2217:
URL: https://github.com/apache/arrow-datafusion/pull/2217#issuecomment-1098090215

   Another physical plan description optimization would be to limit each physical plan's max line width. This would be especially useful for the ParquetExec node:
   
   ```
   ProjectionExec: expr=[ss_sold_date_sk@0 as ss_sold_date_sk, ss_customer_sk@1 as ss_customer_sk]
         FilterExec: ss_sold_date_sk@0 IS NOT NULL
           ParquetExec: limit=None, partitions=[/Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00000-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00023-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00012-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00001-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00024-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00030-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00008-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.
 parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00056-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00010-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00078-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00028-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00016-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00017-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00002-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-bl
 aze-extension/dev/tpcds_1g/store_sales/part-00021-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00027-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00032-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00031-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00034-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00014-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00005-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales
 /part-00026-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00019-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00011-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00013-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00089-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00020-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00029-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00004-8ecba12f-a127-4bca-8d50-f9
 516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00015-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00007-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00018-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00067-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00022-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00009-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00006-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users
 /shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00045-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00033-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00003-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00025-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00035-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00096-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00062-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/d
 ev/tpcds_1g/store_sales/part-00068-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00066-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00072-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00077-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00086-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00061-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00063-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00052-8ec
 ba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00082-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00075-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00087-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00059-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00070-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00053-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00050-8ecba12f-a127-4bca-8d50-f9516808f050-c000
 .snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00084-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00085-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00049-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00080-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00038-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00046-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00074-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/
 spark-blaze-extension/dev/tpcds_1g/store_sales/part-00043-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00076-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00093-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00058-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00039-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00090-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00036-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/sto
 re_sales/part-00083-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00051-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00079-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00092-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00055-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00081-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00088-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00071-8ecba12f-a127-4bca
 -8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00044-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00042-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00069-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00048-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00054-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00095-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00065-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet
 , /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00047-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00094-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00037-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00073-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00064-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00057-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00041-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-ext
 ension/dev/tpcds_1g/store_sales/part-00060-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00091-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00040-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet, /Users/shenyijie/oss/spark-blaze-extension/dev/tpcds_1g/store_sales/part-00097-8ecba12f-a127-4bca-8d50-f9516808f050-c000.snappy.parquet], predicate=true, projection=[ss_sold_date_sk, ss_customer_sk]
   ```
   
   It would be wonderful if I could reduce this to:
   ```
   ParquetExec: limit=None, predicate=true, projection=[ss_sold_date_sk, ss_customer_sk], partitions=[100 items ... ]
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org