You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "himanshu-garg-zepto (via GitHub)" <gi...@apache.org> on 2025/04/10 10:43:32 UTC
[I] [Bug][DBT] Git clone fails on all but the first run because folder already exists [incubator-devlake]
himanshu-garg-zepto opened a new issue, #8377:
URL: https://github.com/apache/incubator-devlake/issues/8377
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
### What happened
When using an advanced blueprint based on the DBT plugin with a remote git project url, the "Git" task of DBT fails on every subsequent run.
It's because it tries to git clone the project which succeeds the first time but fails on every subsequent run due to "folder already exists".
If I manually delete the folder from within the container before running the DBT pipeline again, it works.
### What do you expect to happen
I expect DBT based blueprints to not fail every time after the first
### How to reproduce
- setup devlake
- add any project (github, gitlab etc)
- add a new advanced blueprint with dbt template with remote git url of a dummy dbt project
- run the blueprint (should succeed)
- run the blueprint again (should fail at the Git step)
### Anything else
_No response_
### Version
1.0
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][DBT] Git clone fails on all but the first run because folder already exists [incubator-devlake]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #8377:
URL: https://github.com/apache/incubator-devlake/issues/8377#issuecomment-2792327747
<!-- Greeting -->
Hello @himanshu-garg-zepto! I'm [Dosu](https://dosu.dev), your friendly bot here to help you solve bugs, answer questions, and assist you in becoming a contributor. I'm here while you wait for a human maintainer to jump in. Let's tackle this issue together!
<!-- Answer -->
To address the issue where the "Git" task fails on subsequent runs due to the "folder already exists" error, you can modify the `Git` function to check if the directory already exists before attempting to clone the repository. If the directory exists, you can skip the cloning process. Here is a modified version of the `Git` function:
```go
func Git(taskCtx plugin.SubTaskContext) errors.Error {
logger := taskCtx.GetLogger()
data := taskCtx.GetData().(*DbtTaskData)
if data.Options.ProjectGitURL == "" {
return nil
}
// Check if the directory already exists
projectDir := "your_project_directory" // Replace with the actual directory path
if _, err := os.Stat(projectDir); !os.IsNotExist(err) {
logger.Info("Project directory already exists, skipping clone.")
return nil
}
cmd := exec.Command("git", "clone", data.Options.ProjectGitURL)
logger.Info("start clone dbt project: %v", cmd)
out, err := cmd.CombinedOutput()
if err != nil {
logger.Error(err, "clone dbt project failed")
return errors.Convert(err)
}
logger.Info("clone dbt project success: %v", string(out))
return nil
}
```
This code checks if the project directory already exists and skips the cloning process if it does, thus preventing the error <sup>[[1]](https://github.com/apache/incubator-devlake/blob/main/backend/plugins/dbt/tasks/git.go)</sup>.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/2d0f409e-ee18-4d31-81bb-dfca0f710a55?feedback_type=other)</sup>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][DBT] Git clone fails on all but the first run because folder already exists [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh closed issue #8377: [Bug][DBT] Git clone fails on all but the first run because folder already exists
URL: https://github.com/apache/incubator-devlake/issues/8377
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org