[GitHub] [superset] AAfghahi commented on a change in pull request #15719: feat: adding Progress Bar to Benchmark script

[GitBox <gi...@apache.org>]

AAfghahi commented on a change in pull request #15719:
URL: https://github.com/apache/superset/pull/15719#discussion_r670808443



##########
File path: scripts/benchmark_migration.py
##########
@@ -171,25 +172,29 @@ def main(
 
     min_entities = 10
     new_models: Dict[Type[Model], List[Model]] = defaultdict(list)
+    bar = ChargingBar("Processing", max=min_entities)
     while min_entities <= limit:
         downgrade(revision=down_revision)
         print(f"Running with at least {min_entities} entities of each model")
-        for model in models:
-            missing = min_entities - model_rows[model]
-            if missing > 0:
-                print(f"- Adding {missing} entities to the {model.__name__} model")
-                try:
-                    added_models = add_sample_rows(session, model, missing)
-                except Exception:
-                    session.rollback()
-                    raise
-                model_rows[model] = min_entities
-                session.commit()
-
-                if auto_cleanup:
-                    new_models[model].extend(added_models)
+        for i in range(min_entities):
+            for model in models:
+                missing = min_entities - model_rows[model]
+                if missing > 0:
+                    print(f"- Adding {missing} entities to the {model.__name__} model")
+                    try:
+                        added_models = add_sample_rows(session, model, missing)

Review comment:
       Its totally cool, it was fun to work on this, and to learn how to use dependencies. Should I drop this process because of this complication, or is there a way that we can salvage this/is it worth it. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org