[PR] fix: dbt test dependency refs, CI bootstrap drift, and bootstrap idempotency [fineract-business-intelligence]

["airajena (via GitHub)" <gi...@apache.org>]

airajena opened a new pull request, #69:
URL: https://github.com/apache/fineract-business-intelligence/pull/69

   ## Summary
   
   - Two singular dbt tests referenced `fact_loan_snapshot` and raw tables as plain SQL instead of `ref()`/`source()`, so dbt couldn't schedule them correctly  caused nondeterministic "relation does not exist" errors depending on build thread timing.
   - CI's integration-test job inlined a hand-copied, already-drifted view SQL block instead of calling `bootstrap_source.sh`  now calls the real script, so CI and local dev can never diverge again.
   - `bootstrap_source.sh` / `bootstrap_fineract_source.sh` weren't safely re-runnable: `CREATE OR REPLACE VIEW` can't reorder a view's columns, so re-running against `batch_job_execution` failed. Wrapped in `BEGIN`/`COMMIT` and added `DROP VIEW IF EXISTS` first.
   - `run_pipeline.sh` had no protection against two runs overlapping added a `flock`-based lock, and auto-runs `dbt deps` so a fresh clone doesn't need a manual step.
   - `.gitignore` was missing `.venv/`.
   
   ## Test plan
   
   - [x] `bootstrap_source.sh` run 3x back-to-back : clean, idempotent
   - [x] `dbt build --full-refresh` : 116/116, deterministic across repeated runs
   - [x] Full backfill and incremental extraction verified against a live source DB, including delete propagation
   - [x] Raced two concurrent `run_pipeline.sh` invocations : second correctly rejected by the lock
   - [x] Full pipeline (backfill + incremental) completes end-to-end including Superset refresh
   - [x] RLS integration suite : 46/46 passing
   - [x] shellcheck + sqlfluff clean on all modified files
   - [x] `ci.yml` validated as syntactically correct YAML
   
   Fixes: #68 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix: dbt test dependency refs, CI bootstrap drift, and bootstrap idempotency [fineract-business-intelligence]

["Aman-Mittal (via GitHub)" <gi...@apache.org>]

Aman-Mittal merged PR #69:
URL: https://github.com/apache/fineract-business-intelligence/pull/69


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org