[GitHub] [arrow-datafusion] mustafasrepo commented on issue #7162: Do not sort data that is already sorted

["mustafasrepo (via GitHub)" <gi...@apache.org>]

mustafasrepo commented on issue #7162:
URL: https://github.com/apache/arrow-datafusion/issues/7162#issuecomment-1661651586

   Actually ordering defined at the source is `ORDER ("key" ASC, "time" ASC)`, which lexicographical ordering (tiebreaks are resolved by next expression in the vector).  However, ordering asked in the second query is `order by "time" ASC`.  Table is not globally ordered according to `"time" ASC`. Hence that sort is need in the second query. However if during table creation, query were 
   ```sql
   CREATE EXTERNAL TABLE test5 (
       "key"  VARCHAR NOT NULL,
       "value"  INT NOT NULL,
       "time"  BIGINT NOT NULL
   )
   STORED AS CSV
   WITH ORDER ("key" ASC)
   WITH ORDER ("time" DESC)
   LOCATION 'path/to/your_data.csv';
   ```
   (which according to data at your example is satisfied). Query below will not have `SortExec` in its plan. Because `time DESC` specification at the source is global. 
   ```sql
   explain select key, time, value
   from test5
   where key = 2
   order by "time" DESC
   limit 1;
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org