You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@madlib.apache.org by GitBox <gi...@apache.org> on 2019/09/09 18:58:38 UTC

[GitHub] [madlib] orhankislal commented on a change in pull request #439: DL: Add support in preprocessor to evenly distribute data for GPDB

orhankislal commented on a change in pull request #439: DL: Add support in preprocessor to evenly distribute data for GPDB
URL: https://github.com/apache/madlib/pull/439#discussion_r322392420
 
 

 ##########
 File path: src/ports/postgres/modules/deep_learning/test/input_data_preprocessor.sql_in
 ##########
 @@ -88,6 +89,23 @@ SELECT training_preprocessor_dl(
   'label',
   'x');
 
+-- Test data is evenly distributed across all segments (GPDB only)
+m4_changequote(`<!', `!>')
+m4_ifdef(<!__POSTGRESQL__!>, <!!>, <!
+DROP TABLE IF EXISTS data_preprocessor_input_batch, data_preprocessor_input_batch_summary;
+SELECT training_preprocessor_dl(
+  'data_preprocessor_input',
+  'data_preprocessor_input_batch',
+  'id',
+  'x',
+  1);
+
+SELECT assert(count(*)=(SELECT ceil(17.0/count(*)) from gp_segment_configuration WHERE role = 'p' and content != -1), 'Even distribution of buffers failed.')
+FROM data_preprocessor_input_batch
+WHERE gp_segment_id = 0
+GROUP BY gp_segment_id;
 
 Review comment:
   If we have the `WHERE gp_segment_id = 0` clause, why do we need the `group by`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services