Re: [I] [Ruby] Building an array of hashes from Table is very slow [arrow]

["kou (via GitHub)" <gi...@apache.org>]

kou commented on issue #45117:
URL: https://github.com/apache/arrow/issues/45117#issuecomment-2564593313

   In general, you should not convert large data to raw Ruby objects. It's slow as you seen.
   
   But Red Arrow provides an optimized feature that converts an Arrow array to a Ruby array. You can use it by `arrow::XXXArray#to_a`.
   
   For example, you can get a `Hash` of `Array`s (not an `Array` of `Hash`es) with the feature:
   
   ```ruby
   def read_parquet
     table = Arrow::TableLoader.load('data.parquet', { format: :parquet })
     data = {}
     table.schema.fields.each do |field|
       data[field.name] = []
     end
     table.each_record_batch do |record_batch|
       record_batch.each_column do |column|
         data[column.name].concat(column.data.to_a)
       end
     end
     data
   end
   ```
   
   Are you really need an `Array` of `Hash`es?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org