[GitHub] [arrow-rs] alamb commented on a change in pull request #797: Add Parquet writer example to docs

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #797:
URL: https://github.com/apache/arrow-rs/pull/797#discussion_r714255092



##########
File path: parquet/src/arrow/mod.rs
##########
@@ -79,8 +52,57 @@
 //!     writer.write(&batch).expect("Writing batch");
 //! }
 //! writer.close().unwrap();
+//! ```
+
+//! `WriterProperties` can be used to set several configuration options
+//! ```rust, no_run
+//! use parquet::basic::{ Compression, Encoding };
+//! // File compression
+//! let props = WriterProperties::builder()
+//!     .set_compression(Compression::SNAPPY)
+//!     .build();
+//! // Max row group size compression
+//! let props = WriterProperties::builder()
+//!     .set_max_row_group_size(100)
+//!     .build();
+//! // File encoding
+//! let props = WriterProperties::builder()
+//!     .set_encoding(Encoding::RLE)
+//!     .build();
+//! // Parquet Version
+//! let props = WriterProperties::builder()
+//!     .set_writer_version(WriterVersion::PARQUET_1_0)
+//!     .build();
+//! ```
+//!
+//! # Example of reading parquet file into arrow record batch
+//!
+//! ```rust, no_run
+//! use arrow::record_batch::RecordBatchReader;
+//! use parquet::file::reader::SerializedFileReader;
+//! use parquet::arrow::{ParquetFileArrowReader, ArrowReader};
+//! use std::sync::Arc;
+//! use std::fs::File;
 //!
+//! let file = File::open("data.parquet").unwrap();
+//! let file_reader = SerializedFileReader::new(file).unwrap();

Review comment:
       What do you think about not using a `SerializedFileReader` and instead create a batch like the following (from https://docs.rs/arrow/5.4.0/arrow/record_batch/struct.RecordBatch.html#method.try_from_iter): 
   
   ```
   
   let a: ArrayRef = Arc::new(Int32Array::from(vec![1, 2]));
   let b: ArrayRef = Arc::new(StringArray::from(vec!["a", "b"]));
   
   let record_batch = RecordBatch::try_from_iter(vec![
     ("a", a),
     ("b", b),
   ]);
   ```
   
   I think that would make the example simpler as well as allowing it to be run (you could remove the `no_run`)
   

##########
File path: parquet/src/arrow/mod.rs
##########
@@ -79,8 +52,57 @@
 //!     writer.write(&batch).expect("Writing batch");
 //! }
 //! writer.close().unwrap();
+//! ```
+
+//! `WriterProperties` can be used to set several configuration options
+//! ```rust, no_run
+//! use parquet::basic::{ Compression, Encoding };
+//! // File compression
+//! let props = WriterProperties::builder()
+//!     .set_compression(Compression::SNAPPY)
+//!     .build();
+//! // Max row group size compression
+//! let props = WriterProperties::builder()
+//!     .set_max_row_group_size(100)
+//!     .build();
+//! // File encoding
+//! let props = WriterProperties::builder()
+//!     .set_encoding(Encoding::RLE)
+//!     .build();
+//! // Parquet Version
+//! let props = WriterProperties::builder()
+//!     .set_writer_version(WriterVersion::PARQUET_1_0)
+//!     .build();

Review comment:
       ```suggestion
   ```
   
   I think a single example of changing the writer properties is probably sufficient

##########
File path: parquet/src/arrow/mod.rs
##########
@@ -79,8 +52,57 @@
 //!     writer.write(&batch).expect("Writing batch");
 //! }
 //! writer.close().unwrap();
+//! ```
+
+//! `WriterProperties` can be used to set several configuration options
+//! ```rust, no_run

Review comment:
       I think it would be ok to run this example (and that way this example would get included in the doc tests)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org