[GitHub] [incubator-mxnet] larroy commented on a change in pull request #15249: Sort benchmark output per op name, add some validation to CLI

[GitBox <gi...@apache.org>]

larroy commented on a change in pull request #15249: Sort benchmark output per op name, add some validation to CLI
URL: https://github.com/apache/incubator-mxnet/pull/15249#discussion_r294025110
 
 

 ##########
 File path: benchmark/opperf/opperf.py
 ##########
 @@ -108,26 +113,33 @@ def _parse_mxnet_context(ctx):
                              'Valid Inputs - cpu, gpu, gpu(0), gpu(1)...')
     parser.add_argument('--dtype', type=str, default='float32', help='DType (Precision) to run benchmarks. By default, '
                                                                      'float32. Valid Inputs - float32, float64.')
-    parser.add_argument('--output-format', type=str, default='json',
+    parser.add_argument('-f', '--output-format', type=str, default='json',
+                        choices=['json', 'md'],
                         help='Benchmark result output format. By default, json. '
                              'Valid Inputs - json, md')
 
-    parser.add_argument('--output-file', type=str, default='./mxnet_operator_benchmarks.json',
+    parser.add_argument('-o', '--output-file', type=str, default='./mxnet_operator_benchmarks.json',
                         help='Name and path for the '
                              'output file.')
 
-    user_options = parser.parse_args()
-    logging.info(f"Running MXNet operator benchmarks with the following options: {user_options}")
+    args = parser.parse_args()
+    logging.info(f"Running MXNet operator benchmarks with the following options: {args}")
+    assert not os.path.isfile(args.output_file), f"Output file {args.output_file} already exists."
 
     # 2. RUN BENCHMARKS
-    ctx = _parse_mxnet_context(user_options.ctx)
-    dtype = user_options.dtype
-    final_benchmark_results = run_all_mxnet_operator_benchmarks(ctx=ctx, dtype=user_options.dtype)
+    ctx = _parse_mxnet_context(args.ctx)
+    dtype = args.dtype
+    final_benchmark_results = run_all_mxnet_operator_benchmarks(ctx=ctx, dtype=args.dtype)
 
     # 3. PREPARE OUTPUTS
-    save_to_file(final_benchmark_results, user_options.output_file, user_options.output_format)
+    save_to_file(final_benchmark_results, args.output_file, args.output_format)
 
     # 4. Generate list of MXNet operators not covered in benchmarks
     ops_not_covered = get_operators_with_no_benchmark(final_benchmark_results.keys())
     for idx, op in enumerate(ops_not_covered):
         print(f"{idx}. {op}")
+    return 0
 
 Review comment:
   No, it's because of the wrap in sys.exit below. I always do this in scripts to be more explicit about the return value in error conditions.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services