[GitHub] [arrow] nealrichardson commented on pull request #12826: ARROW-15260: [R] open_dataset - add file_name as column

[GitBox <gi...@apache.org>]

nealrichardson commented on PR #12826:
URL: https://github.com/apache/arrow/pull/12826#issuecomment-1175469687

   I'm missing something here and I can't see what it is. I see that the ScanNode gets the "augmented fields" added to its schema: https://github.com/apache/arrow/blob/master/cpp/src/arrow/dataset/scanner.cc#L924-L932 So subsequent nodes should see them in the `output_schema` and be able to use them. 
   
   If I comment out these lines and don't project them away: https://github.com/apache/arrow/blob/master/r/R/query-engine.R#L150-L151 and I just do `collect(ds)`, I see the augmented fields in the result. But with those lines in place, if I do `mutate(files = Expression$field_ref("__filename"))` and then collect, I get the error that @thisisnic showed above. 
   
   I can't rule out that there's an extra Project happening in R that I'm not seeing somewhere that is dropping the columns. I think I need to implement the ExecPlan::ToString bindings in order to help solve this. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org