You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "pvillard31 (via GitHub)" <gi...@apache.org> on 2026/03/02 10:22:52 UTC
[PR] NIFI-15657 - Fix 409 Conflict in Azure DevOps and Bitbucket Cloud when multiple flows share the same branch [nifi]
pvillard31 opened a new pull request, #10948:
URL: https://github.com/apache/nifi/pull/10948
# Summary
NIFI-15657 - Fix 409 Conflict in Azure DevOps and Bitbucket Cloud when multiple flows share the same branch
- Azure DevOps and Bitbucket Cloud flow registry clients now correctly use the branch HEAD (instead of a file-level commit SHA) for their push APIs' optimistic locking parameters, fixing 409 Conflict errors in monorepo scenarios where multiple independent flows are versioned on the same branch.
- Both providers now implement a retry-with-file-verification loop: on 409, re-verify the target file hasn't changed, re-fetch the branch HEAD, and retry (up to 3 attempts). File-level conflicts still fail immediately.
- Extracted `fetchBranchHead()` helper in AzureDevOpsRepositoryClient (also deduplicates `deleteContent()`), and added `getBranchHeadCloud()` in BitbucketRepositoryClient using the Refs API.
### Root Cause
The Azure DevOps Push API requires `oldObjectId` to be the current branch HEAD, and the Bitbucket Cloud Source API requires `parents` to be the branch HEAD. Both providers were incorrectly passing the file-level commit SHA (`expectedCommitSha` from `GitCreateContentRequest`), which only equals the branch HEAD in single-flow-per-branch setups. In monorepo scenarios, any commit to a different flow file advances the branch HEAD, causing the next flow's commit to fail with 409.
### Note
Azure DevOps and Bitbucket Cloud only offer branch-level optimistic locking in their push APIs, so our file-level conflict check happens client-side one HTTP round-trip before the push, leaving a narrow TOCTOU window. In practice the risk is negligible — the gap is milliseconds, and two users would have to save the exact same flow file within that window. GitHub, GitLab, and Bitbucket DC have no such gap because their APIs provide server-side file-level atomicity.
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
- [ ] Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status
### Pull Request Formatting
- [ ] Pull Request based on current revision of the `main` branch
- [ ] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [ ] Build completed using `./mvnw clean install -P contrib-check`
- [ ] JDK 21
- [ ] JDK 25
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15657 - Fix 409 Conflict in Azure DevOps and Bitbucket Cloud when multiple flows share the same branch [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory merged PR #10948:
URL: https://github.com/apache/nifi/pull/10948
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15657 - Fix 409 Conflict in Azure DevOps and Bitbucket Cloud when multiple flows share the same branch [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #10948:
URL: https://github.com/apache/nifi/pull/10948#discussion_r2914383193
##########
nifi-extension-bundles/nifi-azure-bundle/nifi-azure-registry-clients/src/main/java/org/apache/nifi/azure/devops/AzureDevOpsRepositoryClient.java:
##########
@@ -389,11 +372,59 @@ public String createContent(final GitCreateContentRequest request) throws FlowRe
}
final String encoded = Base64.getEncoder().encodeToString(request.getContent().getBytes(StandardCharsets.UTF_8));
+ final URI pushUri = getUriBuilder().addPathSegment(SEGMENT_PUSHES)
+ .addQueryParameter(API, API_VERSION)
+ .build();
+
+ for (int attempt = 1; attempt <= MAX_PUSH_ATTEMPTS; attempt++) {
+ if (expectedFileCommit != null) {
+ final Optional<String> currentFileCommit = getContentSha(request.getPath(), branch);
+ if (currentFileCommit.isPresent() && !currentFileCommit.get().equals(expectedFileCommit)) {
+ throw new FlowRegistryException("File [%s] has been modified by another commit".formatted(path));
+ }
+ }
+ final String branchHead = fetchBranchHead(branch);
+ final HttpResponseEntity response = executePush(pushUri, branch, branchHead, encoded, message, changeType, path);
+
+ if (response.statusCode() == HttpURLConnection.HTTP_CREATED) {
+ try {
+ final JsonNode pushResponse = MAPPER.readTree(response.body());
+ return pushResponse.get(SEGMENT_COMMITS).get(0).get(JSON_FIELD_COMMIT_ID).asText();
+ } catch (final IOException e) {
+ throw new FlowRegistryException("Failed to parse push response", e);
Review Comment:
It would be helpful to include the `pushUri` in this message.
##########
nifi-extension-bundles/nifi-azure-bundle/nifi-azure-registry-clients/src/main/java/org/apache/nifi/azure/devops/AzureDevOpsRepositoryClient.java:
##########
@@ -389,11 +372,59 @@ public String createContent(final GitCreateContentRequest request) throws FlowRe
}
final String encoded = Base64.getEncoder().encodeToString(request.getContent().getBytes(StandardCharsets.UTF_8));
+ final URI pushUri = getUriBuilder().addPathSegment(SEGMENT_PUSHES)
+ .addQueryParameter(API, API_VERSION)
+ .build();
+
+ for (int attempt = 1; attempt <= MAX_PUSH_ATTEMPTS; attempt++) {
+ if (expectedFileCommit != null) {
+ final Optional<String> currentFileCommit = getContentSha(request.getPath(), branch);
+ if (currentFileCommit.isPresent() && !currentFileCommit.get().equals(expectedFileCommit)) {
+ throw new FlowRegistryException("File [%s] has been modified by another commit".formatted(path));
Review Comment:
It would be helpful to include the current file commit value in the message for reference purposes.
##########
nifi-extension-bundles/nifi-atlassian-bundle/nifi-atlassian-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java:
##########
@@ -427,33 +429,75 @@ public String createContent(final GitCreateContentRequest request) throws FlowRe
}
private String createContentCloud(final GitCreateContentRequest request, final String resolvedPath, final String branch) throws FlowRegistryException {
- final StandardMultipartFormDataStreamBuilder multipartBuilder = new StandardMultipartFormDataStreamBuilder();
- multipartBuilder.addPart(resolvedPath, StandardHttpContentType.APPLICATION_JSON, request.getContent().getBytes(StandardCharsets.UTF_8));
- multipartBuilder.addPart(FIELD_MESSAGE, StandardHttpContentType.TEXT_PLAIN, request.getMessage().getBytes(StandardCharsets.UTF_8));
- multipartBuilder.addPart(FIELD_BRANCH, StandardHttpContentType.TEXT_PLAIN, branch.getBytes(StandardCharsets.UTF_8));
+ final String expectedFileCommit = request.getExpectedCommitSha();
+ final URI uri = getRepositoryUriBuilder().addPathSegment("src").build();
- // Add parents parameter for atomic commit - Bitbucket Cloud will reject if the branch has moved
- final String expectedCommitSha = request.getExpectedCommitSha();
- if (expectedCommitSha != null && !expectedCommitSha.isBlank()) {
- multipartBuilder.addPart(FIELD_PARENTS, StandardHttpContentType.TEXT_PLAIN, expectedCommitSha.getBytes(StandardCharsets.UTF_8));
+ for (int attempt = 1; attempt <= MAX_PUSH_ATTEMPTS; attempt++) {
+ if (expectedFileCommit != null) {
+ final Optional<String> currentFileCommit = getLatestCommit(branch, resolvedPath);
+ if (currentFileCommit.isPresent() && !currentFileCommit.get().equals(expectedFileCommit)) {
+ throw new FlowRegistryException("File [%s] has been modified by another commit".formatted(resolvedPath));
Review Comment:
It would be helpful to include the current file commit is this message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-15657 - Fix 409 Conflict in Azure DevOps and Bitbucket Cloud when multiple flows share the same branch [nifi]
Posted by "pvillard31 (via GitHub)" <gi...@apache.org>.
pvillard31 commented on PR #10948:
URL: https://github.com/apache/nifi/pull/10948#issuecomment-4034507172
I improved the exception messages as you suggested. Thanks @exceptionfactory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org