[GitHub] [arrow] andygrove commented on a change in pull request #7208: ARROW-8833: [Rust] Implement VALIDATE mode in integration tests

[GitBox <gi...@apache.org>]

andygrove commented on a change in pull request #7208:
URL: https://github.com/apache/arrow/pull/7208#discussion_r426271971



##########
File path: rust/integration-testing/src/bin/arrow-json-integration-test.rs
##########
@@ -345,12 +331,57 @@ fn arrow_to_json(
         dictionaries: None,
     };
 
-    let json_file = File::create(json_name).unwrap();
+    let json_file = File::create(json_name)?;
     serde_json::to_writer(&json_file, &arrow_json).unwrap();
 
     Ok(())
 }
 
-fn validate(_arrow_name: &str, _json_name: &str, _verbose: bool) -> Result<(), String> {
-    panic!("validate not implemented");
+fn validate(arrow_name: &str, json_name: &str, _verbose: bool) -> Result<()> {
+    // open JSON file
+    let (json_schema, json_batches) = read_json_file(json_name)?;
+
+    // open Arrow file
+    let arrow_file = File::open(arrow_name)?;
+    let mut arrow_reader = FileReader::try_new(arrow_file)?;
+    let arrow_schema = arrow_reader.schema().as_ref().to_owned();
+
+    // compare schemas
+    assert!(json_schema == arrow_schema);
+
+    for json_batch in &json_batches {
+        if let Some(arrow_batch) = arrow_reader.next_batch()? {
+            // compare batches
+            assert!(arrow_batch.num_columns() == json_batch.num_columns());
+            assert!(arrow_batch.num_rows() == json_batch.num_rows());
+
+        // TODO compare in more detail

Review comment:
       I'm hoping we can have follow up PRs to improve the level of comparison




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org